Jul 17 09:24:55.611062 ip-10-0-136-235 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 17 09:24:55.611073 ip-10-0-136-235 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 17 09:24:55.611080 ip-10-0-136-235 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 17 09:24:55.611308 ip-10-0-136-235 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 17 09:25:05.803476 ip-10-0-136-235 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 17 09:25:05.803493 ip-10-0-136-235 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 72838951eb7c4030b7f9b79b15986c4e --
Jul 17 09:26:40.828566 ip-10-0-136-235 systemd[1]: Starting Kubernetes Kubelet...
Jul 17 09:26:41.227117 ip-10-0-136-235 kubenswrapper[2574]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 09:26:41.227117 ip-10-0-136-235 kubenswrapper[2574]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 17 09:26:41.227117 ip-10-0-136-235 kubenswrapper[2574]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 09:26:41.227117 ip-10-0-136-235 kubenswrapper[2574]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 17 09:26:41.227117 ip-10-0-136-235 kubenswrapper[2574]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 17 09:26:41.229915 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.229826    2574 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 17 09:26:41.236072 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236054    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 09:26:41.236072 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236070    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 09:26:41.236072 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236073    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 09:26:41.236072 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236077    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 09:26:41.236226 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236080    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 09:26:41.236226 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236083    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 09:26:41.236226 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236086    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 09:26:41.236226 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236088    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 09:26:41.236226 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236091    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 09:26:41.236226 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236093    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 09:26:41.236226 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236096    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 09:26:41.236226 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236098    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 09:26:41.236226 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236101    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 09:26:41.236226 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236103    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 09:26:41.236226 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236106    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 09:26:41.236226 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236109    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 09:26:41.236226 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236111    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 09:26:41.236226 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236114    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 09:26:41.236226 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236116    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 09:26:41.236226 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236120    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 09:26:41.236226 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236124    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 09:26:41.236226 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236127    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 09:26:41.236226 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236130    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 09:26:41.236737 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236133    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 09:26:41.236737 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236136    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 09:26:41.236737 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236138    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 09:26:41.236737 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236141    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 09:26:41.236737 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236144    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 09:26:41.236737 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236146    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 09:26:41.236737 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236149    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 09:26:41.236737 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236153    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 09:26:41.236737 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236156    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 09:26:41.236737 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236158    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 09:26:41.236737 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236161    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 09:26:41.236737 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236163    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 09:26:41.236737 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236166    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 09:26:41.236737 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236168    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 09:26:41.236737 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236172    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 09:26:41.236737 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236176    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 09:26:41.236737 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236179    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 09:26:41.236737 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236181    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 09:26:41.236737 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236184    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 09:26:41.236737 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236186    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 09:26:41.237230 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236189    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 09:26:41.237230 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236192    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 09:26:41.237230 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236194    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 09:26:41.237230 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236197    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 09:26:41.237230 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236199    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 09:26:41.237230 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236201    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 09:26:41.237230 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236204    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 09:26:41.237230 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236206    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 09:26:41.237230 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236209    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 09:26:41.237230 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236211    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 09:26:41.237230 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236214    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 09:26:41.237230 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236216    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 09:26:41.237230 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236220    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 09:26:41.237230 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236222    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 09:26:41.237230 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236225    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 09:26:41.237230 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236227    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 09:26:41.237230 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236230    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 09:26:41.237230 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236232    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 09:26:41.237230 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236235    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 09:26:41.237230 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236238    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 09:26:41.237723 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236241    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 09:26:41.237723 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236243    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 09:26:41.237723 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236246    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 09:26:41.237723 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236249    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 09:26:41.237723 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236251    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 09:26:41.237723 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236253    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 09:26:41.237723 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236256    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 09:26:41.237723 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236259    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 09:26:41.237723 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236261    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 09:26:41.237723 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236263    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 09:26:41.237723 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236266    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 09:26:41.237723 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236268    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 09:26:41.237723 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236270    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 09:26:41.237723 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236273    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 09:26:41.237723 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236275    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 09:26:41.237723 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236277    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 09:26:41.237723 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236280    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 09:26:41.237723 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236282    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 09:26:41.237723 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236284    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 09:26:41.237723 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236287    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 09:26:41.238225 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236307    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 09:26:41.238225 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236311    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 09:26:41.238225 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236313    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 09:26:41.238225 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236315    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 09:26:41.238225 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236318    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 09:26:41.238225 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236321    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 09:26:41.238225 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236323    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 09:26:41.238225 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236326    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 09:26:41.238225 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236330    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 09:26:41.238225 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236333    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 09:26:41.238225 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236336    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 09:26:41.238225 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236339    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 09:26:41.238225 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236342    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 09:26:41.238225 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236344    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 09:26:41.238225 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236347    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 09:26:41.238225 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.236349    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 09:26:41.238225 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237048    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 09:26:41.238225 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237055    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 09:26:41.238225 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237058    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 09:26:41.238225 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237061    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 09:26:41.238716 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237063    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 09:26:41.238716 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237066    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 09:26:41.238716 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237069    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 09:26:41.238716 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237071    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 09:26:41.238716 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237074    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 09:26:41.238716 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237076    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 09:26:41.238716 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237079    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 09:26:41.238716 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237082    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 09:26:41.238716 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237084    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 09:26:41.238716 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237086    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 09:26:41.238716 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237089    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 09:26:41.238716 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237091    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 09:26:41.238716 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237095    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 09:26:41.238716 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237098    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 09:26:41.238716 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237100    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 09:26:41.238716 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237103    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 09:26:41.238716 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237105    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 09:26:41.238716 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237108    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 09:26:41.238716 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237110    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 09:26:41.238716 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237112    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 09:26:41.239207 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237115    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 09:26:41.239207 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237117    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 09:26:41.239207 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237120    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 09:26:41.239207 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237123    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 09:26:41.239207 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237126    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 09:26:41.239207 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237128    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 09:26:41.239207 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237130    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 09:26:41.239207 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237133    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 09:26:41.239207 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237135    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 09:26:41.239207 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237138    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 09:26:41.239207 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237140    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 09:26:41.239207 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237143    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 09:26:41.239207 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237146    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 09:26:41.239207 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237148    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 09:26:41.239207 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237151    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 09:26:41.239207 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237153    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 09:26:41.239207 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237155    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 09:26:41.239207 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237158    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 09:26:41.239207 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237160    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 09:26:41.239683 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237162    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 09:26:41.239683 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237165    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 09:26:41.239683 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237168    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 09:26:41.239683 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237170    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 09:26:41.239683 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237172    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 09:26:41.239683 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237175    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 09:26:41.239683 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237177    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 09:26:41.239683 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237181    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 09:26:41.239683 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237183    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 09:26:41.239683 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237186    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 09:26:41.239683 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237188    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 09:26:41.239683 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237191    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 09:26:41.239683 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237193    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 09:26:41.239683 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237195    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 09:26:41.239683 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237198    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 09:26:41.239683 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237201    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 09:26:41.239683 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237204    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 09:26:41.239683 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237206    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 09:26:41.239683 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237209    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 09:26:41.239683 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237211    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 09:26:41.240170 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237214    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 09:26:41.240170 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237216    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 09:26:41.240170 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237219    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 09:26:41.240170 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237222    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 09:26:41.240170 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237224    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 09:26:41.240170 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237227    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 09:26:41.240170 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237229    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 09:26:41.240170 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237234    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 09:26:41.240170 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237237    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 09:26:41.240170 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237240    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 09:26:41.240170 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237243    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 09:26:41.240170 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237247    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 09:26:41.240170 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237249    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 09:26:41.240170 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237252    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 09:26:41.240170 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237255    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 09:26:41.240170 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237258    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 09:26:41.240170 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237261    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 09:26:41.240170 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237264    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 09:26:41.240170 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237267    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 09:26:41.240170 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237270    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 09:26:41.240696 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237273    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 09:26:41.240696 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237276    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 09:26:41.240696 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237279    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 09:26:41.240696 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237284    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 09:26:41.240696 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237287    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 09:26:41.240696 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237290    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 09:26:41.240696 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237312    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 09:26:41.240696 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237318    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 09:26:41.240696 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237330    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 09:26:41.240696 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237334    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 09:26:41.240696 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237338    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 09:26:41.240696 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237341    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 09:26:41.240696 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237343    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 09:26:41.240696 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237346    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 09:26:41.240696 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237348    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 09:26:41.240696 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237351    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 09:26:41.240696 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237718    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 09:26:41.240696 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237723    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 09:26:41.240696 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237725    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 09:26:41.241164 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237728    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 09:26:41.241164 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237731    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 09:26:41.241164 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237733    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 09:26:41.241164 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237736    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 09:26:41.241164 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237739    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 09:26:41.241164 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237741    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 09:26:41.241164 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237744    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 09:26:41.241164 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237746    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 09:26:41.241164 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237749    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 09:26:41.241164 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237752    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 09:26:41.241164 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237754    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 09:26:41.241164 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237757    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 09:26:41.241164 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237759    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 09:26:41.241164 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237762    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 09:26:41.241164 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237764    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 09:26:41.241164 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237767    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 09:26:41.241164 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237769    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 09:26:41.241164 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237772    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 09:26:41.241164 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237774    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 09:26:41.241164 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237777    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 09:26:41.241657 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237779    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 09:26:41.241657 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237782    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 09:26:41.241657 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237785    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 09:26:41.241657 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237787    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 09:26:41.241657 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237790    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 09:26:41.241657 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237793    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 09:26:41.241657 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237795    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 09:26:41.241657 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237797    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 09:26:41.241657 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237800    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 09:26:41.241657 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237802    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 09:26:41.241657 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237806    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 09:26:41.241657 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237810    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 09:26:41.241657 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237812    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 09:26:41.241657 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237816    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 09:26:41.241657 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237818    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 09:26:41.241657 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237821    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 09:26:41.241657 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237824    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 09:26:41.241657 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237826    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 09:26:41.241657 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237829    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 09:26:41.242147 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237831    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 09:26:41.242147 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237835    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 09:26:41.242147 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237837    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 09:26:41.242147 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237839    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 09:26:41.242147 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237842    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 09:26:41.242147 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237844    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 09:26:41.242147 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237847    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 09:26:41.242147 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237852    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 09:26:41.242147 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237855    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 09:26:41.242147 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237857    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 09:26:41.242147 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237860    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 09:26:41.242147 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237862    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 09:26:41.242147 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237865    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 09:26:41.242147 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237868    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 09:26:41.242147 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237870    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 09:26:41.242147 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237873    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 09:26:41.242147 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237877    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 09:26:41.242147 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237880    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 09:26:41.242147 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237882    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 09:26:41.242147 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237885    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 09:26:41.242682 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237887    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 09:26:41.242682 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237890    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 09:26:41.242682 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237892    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 09:26:41.242682 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237894    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 09:26:41.242682 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237897    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 09:26:41.242682 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237899    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 09:26:41.242682 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237902    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 09:26:41.242682 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237904    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 09:26:41.242682 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237906    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 09:26:41.242682 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237909    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 09:26:41.242682 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237912    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 09:26:41.242682 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237914    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 09:26:41.242682 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237916    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 09:26:41.242682 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237919    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 09:26:41.242682 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237921    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 09:26:41.242682 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237923    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 09:26:41.242682 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237926    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 09:26:41.242682 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237928    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 09:26:41.242682 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237930    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 09:26:41.243133 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237933    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 09:26:41.243133 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237941    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 09:26:41.243133 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237944    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 09:26:41.243133 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237949    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 09:26:41.243133 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237951    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 09:26:41.243133 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237954    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 09:26:41.243133 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237956    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 09:26:41.243133 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237958    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 09:26:41.243133 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237961    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 09:26:41.243133 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237963    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 09:26:41.243133 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237966    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 09:26:41.243133 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237968    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 09:26:41.243133 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237970    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 09:26:41.243133 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237973    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 09:26:41.243133 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237975    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 09:26:41.243133 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237978    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 09:26:41.243133 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237980    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 09:26:41.243133 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.237984    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 09:26:41.243133 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238053    2574 flags.go:64] FLAG: --address="0.0.0.0"
Jul 17 09:26:41.243133 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238060    2574 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 17 09:26:41.243133 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238066    2574 flags.go:64] FLAG: --anonymous-auth="true"
Jul 17 09:26:41.243710 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238069    2574 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 17 09:26:41.243710 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238073    2574 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 17 09:26:41.243710 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238075    2574 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 17 09:26:41.243710 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238079    2574 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 17 09:26:41.243710 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238083    2574 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 17 09:26:41.243710 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238086    2574 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 17 09:26:41.243710 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238089    2574 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 17 09:26:41.243710 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238092    2574 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 17 09:26:41.243710 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238096    2574 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 17 09:26:41.243710 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238098    2574 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 17 09:26:41.243710 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238101    2574 flags.go:64] FLAG: --cgroup-root=""
Jul 17 09:26:41.243710 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238104    2574 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 17 09:26:41.243710 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238106    2574 flags.go:64] FLAG: --client-ca-file=""
Jul 17 09:26:41.243710 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238110    2574 flags.go:64] FLAG: --cloud-provider="external"
Jul 17 09:26:41.243710 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238113    2574 flags.go:64] FLAG: --cluster-dns="[]"
Jul 17 09:26:41.243710 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238117    2574 flags.go:64] FLAG: --cluster-domain=""
Jul 17 09:26:41.243710 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238120    2574 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 17 09:26:41.243710 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238123    2574 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 17 09:26:41.243710 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238126    2574 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 17 09:26:41.243710 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238129    2574 flags.go:64] FLAG: --container-log-max-files="5"
Jul 17 09:26:41.243710 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238133    2574 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 17 09:26:41.243710 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238136    2574 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 17 09:26:41.243710 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238139    2574 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 17 09:26:41.244248 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238142    2574 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 17 09:26:41.244248 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238145    2574 flags.go:64] FLAG: --contention-profiling="false"
Jul 17 09:26:41.244248 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238147    2574 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 17 09:26:41.244248 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238150    2574 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 17 09:26:41.244248 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238153    2574 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 17 09:26:41.244248 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238156    2574 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 17 09:26:41.244248 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238160    2574 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 17 09:26:41.244248 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238163    2574 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 17 09:26:41.244248 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238166    2574 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 17 09:26:41.244248 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238169    2574 flags.go:64] FLAG: --enable-load-reader="false"
Jul 17 09:26:41.244248 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238171    2574 flags.go:64] FLAG: --enable-server="true"
Jul 17 09:26:41.244248 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238174    2574 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 17 09:26:41.244248 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238179    2574 flags.go:64] FLAG: --event-burst="100"
Jul 17 09:26:41.244248 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238182    2574 flags.go:64] FLAG: --event-qps="50"
Jul 17 09:26:41.244248 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238189    2574 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 17 09:26:41.244248 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238192    2574 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 17 09:26:41.244248 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238194    2574 flags.go:64] FLAG: --eviction-hard=""
Jul 17 09:26:41.244248 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238198    2574 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 17 09:26:41.244248 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238201    2574 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 17 09:26:41.244248 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238203    2574 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 17 09:26:41.244248 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238206    2574 flags.go:64] FLAG: --eviction-soft=""
Jul 17 09:26:41.244248 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238209    2574 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 17 09:26:41.244248 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238212    2574 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 17 09:26:41.244248 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238215    2574 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 17 09:26:41.244248 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238218    2574 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 17 09:26:41.245005 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238221    2574 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 17 09:26:41.245005 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238223    2574 flags.go:64] FLAG: --fail-swap-on="true"
Jul 17 09:26:41.245005 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238227    2574 flags.go:64] FLAG: --feature-gates=""
Jul 17 09:26:41.245005 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238230    2574 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 17 09:26:41.245005 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238233    2574 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 17 09:26:41.245005 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238236    2574 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 17 09:26:41.245005 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238239    2574 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 17 09:26:41.245005 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238242    2574 flags.go:64] FLAG: --healthz-port="10248"
Jul 17 09:26:41.245005 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238244    2574 flags.go:64] FLAG: --help="false"
Jul 17 09:26:41.245005 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238247    2574 flags.go:64] FLAG: --hostname-override="ip-10-0-136-235.ec2.internal"
Jul 17 09:26:41.245005 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238250    2574 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 17 09:26:41.245005 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238253    2574 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 17 09:26:41.245005 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238255    2574 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 17 09:26:41.245005 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238259    2574 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 17 09:26:41.245005 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238262    2574 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 17 09:26:41.245005 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238265    2574 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 17 09:26:41.245005 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238267    2574 flags.go:64] FLAG: --image-service-endpoint=""
Jul 17 09:26:41.245005 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238270    2574 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 17 09:26:41.245005 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238272    2574 flags.go:64] FLAG: --kube-api-burst="100"
Jul 17 09:26:41.245005 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238275    2574 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 17 09:26:41.245005 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238278    2574 flags.go:64] FLAG: --kube-api-qps="50"
Jul 17 09:26:41.245005 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238283    2574 flags.go:64] FLAG: --kube-reserved=""
Jul 17 09:26:41.245005 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238286    2574 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 17 09:26:41.245005 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238289    2574 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 17 09:26:41.245606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238305    2574 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 17 09:26:41.245606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238308    2574 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 17 09:26:41.245606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238311    2574 flags.go:64] FLAG: --lock-file=""
Jul 17 09:26:41.245606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238314    2574 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 17 09:26:41.245606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238316    2574 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 17 09:26:41.245606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238319    2574 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 17 09:26:41.245606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238324    2574 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 17 09:26:41.245606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238327    2574 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 17 09:26:41.245606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238330    2574 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 17 09:26:41.245606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238333    2574 flags.go:64] FLAG: --logging-format="text"
Jul 17 09:26:41.245606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238336    2574 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 17 09:26:41.245606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238339    2574 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 17 09:26:41.245606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238341    2574 flags.go:64] FLAG: --manifest-url=""
Jul 17 09:26:41.245606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238344    2574 flags.go:64] FLAG: --manifest-url-header=""
Jul 17 09:26:41.245606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238348    2574 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 17 09:26:41.245606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238351    2574 flags.go:64] FLAG: --max-open-files="1000000"
Jul 17 09:26:41.245606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238355    2574 flags.go:64] FLAG: --max-pods="110"
Jul 17 09:26:41.245606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238357    2574 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 17 09:26:41.245606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238360    2574 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 17 09:26:41.245606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238363    2574 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 17 09:26:41.245606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238366    2574 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 17 09:26:41.245606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238369    2574 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 17 09:26:41.245606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238371    2574 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 17 09:26:41.245606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238374    2574 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 17 09:26:41.245606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238381    2574 flags.go:64] FLAG: --node-status-max-images="50"
Jul 17 09:26:41.246197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238384    2574 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 17 09:26:41.246197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238387    2574 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 17 09:26:41.246197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238390    2574 flags.go:64] FLAG: --pod-cidr=""
Jul 17 09:26:41.246197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238393    2574 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 17 09:26:41.246197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238399    2574 flags.go:64] FLAG: --pod-manifest-path=""
Jul 17 09:26:41.246197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238403    2574 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 17 09:26:41.246197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238406    2574 flags.go:64] FLAG: --pods-per-core="0"
Jul 17 09:26:41.246197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238409    2574 flags.go:64] FLAG: --port="10250"
Jul 17 09:26:41.246197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238412    2574 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 17 09:26:41.246197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238414    2574 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01071f1d647d41678"
Jul 17 09:26:41.246197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238418    2574 flags.go:64] FLAG: --qos-reserved=""
Jul 17 09:26:41.246197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238420    2574 flags.go:64] FLAG: --read-only-port="10255"
Jul 17 09:26:41.246197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238423    2574 flags.go:64] FLAG: --register-node="true"
Jul 17 09:26:41.246197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238426    2574 flags.go:64] FLAG: --register-with-taints=""
Jul 17 09:26:41.246197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238429    2574 flags.go:64] FLAG: --registry-burst="10"
Jul 17 09:26:41.246197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238435    2574 flags.go:64] FLAG: --registry-qps="5"
Jul 17 09:26:41.246197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238438    2574 flags.go:64] FLAG: --reserved-cpus=""
Jul 17 09:26:41.246197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238441    2574 flags.go:64] FLAG: --reserved-memory=""
Jul 17 09:26:41.246197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238444    2574 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 17 09:26:41.246197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238447    2574 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 17 09:26:41.246197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238450    2574 flags.go:64] FLAG: --rotate-certificates="false"
Jul 17 09:26:41.246197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238452    2574 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 17 09:26:41.246197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238455    2574 flags.go:64] FLAG: --runonce="false"
Jul 17 09:26:41.246197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238458    2574 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 17 09:26:41.246197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238460    2574 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 17 09:26:41.246818 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238463    2574 flags.go:64] FLAG: --seccomp-default="false"
Jul 17 09:26:41.246818 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238466    2574 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 17 09:26:41.246818 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238469    2574 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 17 09:26:41.246818 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238472    2574 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 17 09:26:41.246818 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238474    2574 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 17 09:26:41.246818 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238477    2574 flags.go:64] FLAG: --storage-driver-password="root"
Jul 17 09:26:41.246818 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238480    2574 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 17 09:26:41.246818 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238483    2574 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 17 09:26:41.246818 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238485    2574 flags.go:64] FLAG: --storage-driver-user="root"
Jul 17 09:26:41.246818 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238488    2574 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 17 09:26:41.246818 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238491    2574 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 17 09:26:41.246818 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238495    2574 flags.go:64] FLAG: --system-cgroups=""
Jul 17 09:26:41.246818 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238500    2574 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 17 09:26:41.246818 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238505    2574 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 17 09:26:41.246818 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238508    2574 flags.go:64] FLAG: --tls-cert-file=""
Jul 17 09:26:41.246818 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238510    2574 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 17 09:26:41.246818 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238514    2574 flags.go:64] FLAG: --tls-min-version=""
Jul 17 09:26:41.246818 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238517    2574 flags.go:64] FLAG: --tls-private-key-file=""
Jul 17 09:26:41.246818 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238519    2574 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 17 09:26:41.246818 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238522    2574 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 17 09:26:41.246818 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238525    2574 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 17 09:26:41.246818 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238528    2574 flags.go:64] FLAG: --v="2"
Jul 17 09:26:41.246818 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238531    2574 flags.go:64] FLAG: --version="false"
Jul 17 09:26:41.246818 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238535    2574 flags.go:64] FLAG: --vmodule=""
Jul 17 09:26:41.246818 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238538    2574 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 17 09:26:41.247489 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.238541    2574 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 17 09:26:41.247489 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238661    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 09:26:41.247489 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238665    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 09:26:41.247489 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238668    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 09:26:41.247489 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238671    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 09:26:41.247489 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238674    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 09:26:41.247489 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238677    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 09:26:41.247489 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238679    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 09:26:41.247489 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238682    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 09:26:41.247489 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238685    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 09:26:41.247489 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238687    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 09:26:41.247489 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238690    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 09:26:41.247489 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238692    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 09:26:41.247489 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238695    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 09:26:41.247489 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238697    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 09:26:41.247489 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238700    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 09:26:41.247489 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238702    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 09:26:41.247489 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238705    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 09:26:41.247489 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238709    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 09:26:41.247489 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238713    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 09:26:41.247489 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238716    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 09:26:41.247992 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238718    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 09:26:41.247992 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238721    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 09:26:41.247992 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238724    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 09:26:41.247992 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238726    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 09:26:41.247992 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238729    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 09:26:41.247992 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238731    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 09:26:41.247992 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238736    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 09:26:41.247992 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238739    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 09:26:41.247992 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238742    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 09:26:41.247992 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238745    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 09:26:41.247992 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238753    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 09:26:41.247992 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238756    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 09:26:41.247992 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238759    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 09:26:41.247992 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238761    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 09:26:41.247992 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238764    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 09:26:41.247992 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238767    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 09:26:41.247992 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238769    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 09:26:41.247992 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238772    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 09:26:41.247992 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238775    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 09:26:41.248464 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238777    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 09:26:41.248464 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238781    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 09:26:41.248464 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238785    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 09:26:41.248464 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238788    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 09:26:41.248464 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238790    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 09:26:41.248464 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238793    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 09:26:41.248464 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238795    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 09:26:41.248464 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238798    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 09:26:41.248464 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238801    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 09:26:41.248464 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238803    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 09:26:41.248464 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238807    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 09:26:41.248464 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238810    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 09:26:41.248464 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238813    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 09:26:41.248464 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238815    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 09:26:41.248464 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238818    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 09:26:41.248464 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238820    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 09:26:41.248464 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238823    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 09:26:41.248464 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238825    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 09:26:41.248464 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238828    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 09:26:41.248464 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238831    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 09:26:41.248956 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238833    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 09:26:41.248956 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238835    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 09:26:41.248956 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238838    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 09:26:41.248956 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238840    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 09:26:41.248956 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238847    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 09:26:41.248956 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238851    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 09:26:41.248956 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238855    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 09:26:41.248956 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238858    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 09:26:41.248956 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238861    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 09:26:41.248956 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238863    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 09:26:41.248956 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238866    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 09:26:41.248956 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238869    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 09:26:41.248956 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238871    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 09:26:41.248956 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238874    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 09:26:41.248956 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238876    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 09:26:41.248956 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238879    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 09:26:41.248956 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238881    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 09:26:41.248956 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238884    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 09:26:41.248956 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238887    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 09:26:41.249430 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238889    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 09:26:41.249430 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238891    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 09:26:41.249430 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238894    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 09:26:41.249430 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238898    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 09:26:41.249430 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238902    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 09:26:41.249430 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238904    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 09:26:41.249430 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238907    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 09:26:41.249430 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238909    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 09:26:41.249430 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238912    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 09:26:41.249430 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238914    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 09:26:41.249430 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238916    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 09:26:41.249430 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238919    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 09:26:41.249430 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238921    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 09:26:41.249430 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238924    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 09:26:41.249430 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238926    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 09:26:41.249430 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238929    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 09:26:41.249430 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238931    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 09:26:41.249430 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238934    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 09:26:41.249430 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238937    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 09:26:41.249884 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238939    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 09:26:41.249884 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.238942    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 09:26:41.249884 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.239708    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 09:26:41.249884 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.246319    2574 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 17 09:26:41.249884 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.246333    2574 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 17 09:26:41.249884 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.246351    2574 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 17 09:26:41.249884 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.246357    2574 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 17 09:26:41.249884 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246410    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 09:26:41.249884 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246414    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 09:26:41.249884 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246417    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 09:26:41.249884 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246420    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 09:26:41.249884 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246423    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 09:26:41.249884 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246426    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 09:26:41.249884 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246428    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 09:26:41.249884 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246431    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 09:26:41.250253 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246433    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 09:26:41.250253 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246436    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 09:26:41.250253 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246439    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 09:26:41.250253 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246443    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 09:26:41.250253 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246446    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 09:26:41.250253 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246449    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 09:26:41.250253 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246452    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 09:26:41.250253 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246455    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 09:26:41.250253 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246457    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 09:26:41.250253 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246459    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 09:26:41.250253 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246462    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 09:26:41.250253 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246464    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 09:26:41.250253 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246467    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 09:26:41.250253 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246469    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 09:26:41.250253 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246472    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 09:26:41.250253 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246474    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 09:26:41.250253 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246477    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 09:26:41.250253 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246480    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 09:26:41.250253 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246483    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 09:26:41.250713 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246487    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 09:26:41.250713 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246490    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 09:26:41.250713 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246493    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 09:26:41.250713 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246496    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 09:26:41.250713 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246498    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 09:26:41.250713 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246501    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 09:26:41.250713 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246503    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 09:26:41.250713 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246506    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 09:26:41.250713 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246508    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 09:26:41.250713 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246510    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 09:26:41.250713 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246513    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 09:26:41.250713 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246515    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 09:26:41.250713 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246518    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 09:26:41.250713 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246520    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 09:26:41.250713 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246522    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 09:26:41.250713 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246525    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 09:26:41.250713 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246527    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 09:26:41.250713 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246530    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 09:26:41.250713 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246533    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 09:26:41.250713 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246535    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 09:26:41.251184 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246537    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 09:26:41.251184 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246540    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 09:26:41.251184 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246542    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 09:26:41.251184 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246544    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 09:26:41.251184 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246547    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 09:26:41.251184 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246549    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 09:26:41.251184 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246552    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 09:26:41.251184 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246554    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 09:26:41.251184 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246556    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 09:26:41.251184 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246559    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 09:26:41.251184 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246561    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 09:26:41.251184 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246564    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 09:26:41.251184 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246566    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 09:26:41.251184 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246568    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 09:26:41.251184 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246571    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 09:26:41.251184 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246574    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 09:26:41.251184 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246576    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 09:26:41.251184 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246578    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 09:26:41.251184 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246581    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 09:26:41.251184 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246583    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 09:26:41.251763 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246586    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 09:26:41.251763 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246588    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 09:26:41.251763 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246591    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 09:26:41.251763 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246593    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 09:26:41.251763 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246595    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 09:26:41.251763 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246598    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 09:26:41.251763 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246600    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 09:26:41.251763 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246603    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 09:26:41.251763 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246605    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 09:26:41.251763 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246607    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 09:26:41.251763 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246609    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 09:26:41.251763 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246612    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 09:26:41.251763 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246614    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 09:26:41.251763 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246618    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 09:26:41.251763 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246622    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 09:26:41.251763 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246624    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 09:26:41.251763 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246627    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 09:26:41.251763 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246630    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 09:26:41.251763 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246633    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 09:26:41.251763 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246636    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 09:26:41.252236 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246638    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 09:26:41.252236 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246640    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 09:26:41.252236 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246643    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 09:26:41.252236 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246645    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 09:26:41.252236 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246648    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 09:26:41.252236 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246650    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 09:26:41.252236 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246652    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 09:26:41.252236 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246655    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 09:26:41.252236 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246658    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 09:26:41.252236 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246660    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 09:26:41.252236 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246663    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 09:26:41.252236 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246665    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 09:26:41.252236 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.246670    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 09:26:41.252236 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246787    2574 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 17 09:26:41.252236 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246792    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 17 09:26:41.252606 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246796    2574 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 17 09:26:41.252606 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246800    2574 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 17 09:26:41.252606 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246803    2574 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 17 09:26:41.252606 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246807    2574 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 17 09:26:41.252606 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246810    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 17 09:26:41.252606 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246813    2574 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 17 09:26:41.252606 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246816    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 17 09:26:41.252606 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246819    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 17 09:26:41.252606 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246822    2574 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 17 09:26:41.252606 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246825    2574 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 17 09:26:41.252606 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246828    2574 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 17 09:26:41.252606 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246830    2574 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 17 09:26:41.252606 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246833    2574 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 17 09:26:41.252606 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246836    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 17 09:26:41.252606 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246839    2574 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 17 09:26:41.252606 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246841    2574 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 17 09:26:41.252606 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246844    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 17 09:26:41.252606 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246846    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 17 09:26:41.252606 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246848    2574 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 17 09:26:41.253062 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246851    2574 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 17 09:26:41.253062 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246853    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 17 09:26:41.253062 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246856    2574 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 17 09:26:41.253062 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246858    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 17 09:26:41.253062 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246860    2574 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 17 09:26:41.253062 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246863    2574 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 17 09:26:41.253062 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246866    2574 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 17 09:26:41.253062 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246868    2574 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 17 09:26:41.253062 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246871    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 17 09:26:41.253062 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246873    2574 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 17 09:26:41.253062 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246877    2574 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 17 09:26:41.253062 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246880    2574 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 17 09:26:41.253062 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246883    2574 feature_gate.go:328] unrecognized feature gate: Example2
Jul 17 09:26:41.253062 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246885    2574 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 17 09:26:41.253062 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246888    2574 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 17 09:26:41.253062 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246891    2574 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 17 09:26:41.253062 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246893    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 17 09:26:41.253062 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246896    2574 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 17 09:26:41.253062 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246898    2574 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 17 09:26:41.253062 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246900    2574 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 17 09:26:41.253540 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246903    2574 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 17 09:26:41.253540 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246905    2574 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 17 09:26:41.253540 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246907    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 17 09:26:41.253540 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246910    2574 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 17 09:26:41.253540 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246913    2574 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 17 09:26:41.253540 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246915    2574 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 17 09:26:41.253540 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246917    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 17 09:26:41.253540 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246920    2574 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 17 09:26:41.253540 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246923    2574 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 17 09:26:41.253540 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246925    2574 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 17 09:26:41.253540 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246928    2574 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 17 09:26:41.253540 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246930    2574 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 17 09:26:41.253540 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246932    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 17 09:26:41.253540 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246935    2574 feature_gate.go:328] unrecognized feature gate: Example
Jul 17 09:26:41.253540 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246937    2574 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 17 09:26:41.253540 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246940    2574 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 17 09:26:41.253540 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246942    2574 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 17 09:26:41.253540 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246945    2574 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 17 09:26:41.253540 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246947    2574 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 17 09:26:41.253540 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246950    2574 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 17 09:26:41.254007 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246953    2574 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 17 09:26:41.254007 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246955    2574 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 17 09:26:41.254007 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246957    2574 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 17 09:26:41.254007 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246960    2574 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 17 09:26:41.254007 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246962    2574 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 17 09:26:41.254007 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246964    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 17 09:26:41.254007 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246967    2574 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 17 09:26:41.254007 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246969    2574 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 17 09:26:41.254007 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246972    2574 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 17 09:26:41.254007 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246974    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 17 09:26:41.254007 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246976    2574 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 17 09:26:41.254007 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246979    2574 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 17 09:26:41.254007 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246981    2574 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 17 09:26:41.254007 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246984    2574 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 17 09:26:41.254007 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246986    2574 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 17 09:26:41.254007 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246989    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 17 09:26:41.254007 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246991    2574 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 17 09:26:41.254007 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246993    2574 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 17 09:26:41.254007 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246996    2574 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 17 09:26:41.254007 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.246998    2574 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 17 09:26:41.254499 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.247000    2574 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 17 09:26:41.254499 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.247003    2574 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 17 09:26:41.254499 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.247005    2574 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 17 09:26:41.254499 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.247007    2574 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 17 09:26:41.254499 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.247010    2574 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 17 09:26:41.254499 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.247012    2574 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 17 09:26:41.254499 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.247014    2574 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 17 09:26:41.254499 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.247017    2574 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 17 09:26:41.254499 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.247019    2574 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 17 09:26:41.254499 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.247022    2574 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 17 09:26:41.254499 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.247024    2574 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 17 09:26:41.254499 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.247026    2574 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 17 09:26:41.254499 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.247028    2574 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 17 09:26:41.254499 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.247031    2574 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 17 09:26:41.254499 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.247034    2574 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 17 09:26:41.254499 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.247036    2574 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 17 09:26:41.254499 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.247038    2574 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 17 09:26:41.254499 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:41.247041    2574 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 17 09:26:41.254923 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.247047    2574 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 17 09:26:41.254923 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.247137    2574 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 17 09:26:41.254923 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.249249    2574 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 17 09:26:41.254923 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.250823    2574 server.go:1019] "Starting client certificate rotation"
Jul 17 09:26:41.254923 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.250923    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 17 09:26:41.254923 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.250951    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 17 09:26:41.254923 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.253611    2574 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 17 09:26:41.257374 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.257353    2574 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 17 09:26:41.265985 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.265968    2574 log.go:25] "Validated CRI v1 runtime API"
Jul 17 09:26:41.269967 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.269954    2574 log.go:25] "Validated CRI v1 image API"
Jul 17 09:26:41.271546 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.271531    2574 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 17 09:26:41.274987 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.274964    2574 fs.go:135] Filesystem UUIDs: map[479990d6-b687-46c1-a139-32cf525d9589:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 ae82acd6-dcf7-4a65-b4ce-8342c36e6832:/dev/nvme0n1p4]
Jul 17 09:26:41.275056 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.274987    2574 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 17 09:26:41.279897 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.279796    2574 manager.go:216] Machine: {Timestamp:2026-07-17 09:26:41.278427365 +0000 UTC m=+0.334010241 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3107380 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2fafef4ead8b349d46301515479e47 SystemUUID:ec2fafef-4ead-8b34-9d46-301515479e47 BootID:72838951-eb7c-4030-b7f9-b79b15986c4e Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e8:37:7c:a0:09 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e8:37:7c:a0:09 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9e:69:2e:d4:8f:bc Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 17 09:26:41.279897 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.279892    2574 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 17 09:26:41.280005 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.279957    2574 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 17 09:26:41.281658 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.281644    2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 17 09:26:41.283438 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.283400    2574 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 17 09:26:41.283663 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.283448    2574 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-235.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 17 09:26:41.284416 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.284401    2574 topology_manager.go:138] "Creating topology manager with none policy"
Jul 17 09:26:41.284455 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.284423    2574 container_manager_linux.go:309] "Creating device plugin manager"
Jul 17 09:26:41.284455 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.284434    2574 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 17 09:26:41.285268 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.285257    2574 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 17 09:26:41.286969 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.286959    2574 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 17 09:26:41.290404 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.290393    2574 state_mem.go:36] "Initialized new in-memory state store"
Jul 17 09:26:41.290516 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.290506    2574 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 17 09:26:41.290553 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.290535    2574 kubelet.go:486] "Attempting to sync node with API server"
Jul 17 09:26:41.290553 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.290544    2574 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 17 09:26:41.290618 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.290560    2574 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 17 09:26:41.290618 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.290569    2574 kubelet.go:398] "Adding apiserver pod source"
Jul 17 09:26:41.290618 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.290577    2574 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 17 09:26:41.290867 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.290856    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 17 09:26:41.290901 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.290874    2574 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 17 09:26:41.293312 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.293282    2574 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 17 09:26:41.295052 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.295038    2574 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 17 09:26:41.295113 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.295069    2574 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 17 09:26:41.296001 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.295985    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 17 09:26:41.296040 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.296011    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 17 09:26:41.296040 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.296018    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 17 09:26:41.296040 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.296023    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 17 09:26:41.296040 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.296031    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 17 09:26:41.296040 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.296037    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 17 09:26:41.296200 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.296044    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 17 09:26:41.296200 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.296050    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 17 09:26:41.296200 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.296058    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 17 09:26:41.296200 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.296078    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 17 09:26:41.296200 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.296095    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 17 09:26:41.296200 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.296100    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 17 09:26:41.296956 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.296945    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 17 09:26:41.297006 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.296958    2574 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 17 09:26:41.299988 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.299972    2574 server.go:1268] "Started kubelet"
Jul 17 09:26:41.300124 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.300058    2574 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 17 09:26:41.300124 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.300088    2574 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 17 09:26:41.300213 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.300145    2574 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 17 09:26:41.300648 ip-10-0-136-235 systemd[1]: Started Kubernetes Kubelet.
Jul 17 09:26:41.305139 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.305120    2574 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 17 09:26:41.307377 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.307357    2574 server.go:310] "Adding debug handlers to kubelet server"
Jul 17 09:26:41.307496 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.307456    2574 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 17 09:26:41.307496 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:41.307472    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 17 09:26:41.307624 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:41.307511    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 17 09:26:41.307896 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:41.307879    2574 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 17 09:26:41.311886 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:41.310778    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-235.ec2.internal.18c309924b2cb2af  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-235.ec2.internal,UID:ip-10-0-136-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-235.ec2.internal,},FirstTimestamp:2026-07-17 09:26:41.299944111 +0000 UTC m=+0.355526987,LastTimestamp:2026-07-17 09:26:41.299944111 +0000 UTC m=+0.355526987,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-235.ec2.internal,}"
Jul 17 09:26:41.313691 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.313677    2574 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 17 09:26:41.313691 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.313683    2574 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 17 09:26:41.313858 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:41.313835    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-235.ec2.internal\" not found"
Jul 17 09:26:41.314505 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.314489    2574 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 17 09:26:41.314596 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.314517    2574 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 17 09:26:41.314596 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.314495    2574 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 17 09:26:41.314673 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.314629    2574 reconstruct.go:97] "Volume reconstruction finished"
Jul 17 09:26:41.314673 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.314637    2574 reconciler.go:29] "Reconciler: start to sync state"
Jul 17 09:26:41.319529 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.319514    2574 factory.go:55] Registering systemd factory
Jul 17 09:26:41.319529 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.319531    2574 factory.go:223] Registration of the systemd container factory successfully
Jul 17 09:26:41.319777 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.319757    2574 factory.go:153] Registering CRI-O factory
Jul 17 09:26:41.319777 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.319778    2574 factory.go:223] Registration of the crio container factory successfully
Jul 17 09:26:41.319931 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.319825    2574 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 17 09:26:41.319931 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.319850    2574 factory.go:103] Registering Raw factory
Jul 17 09:26:41.319931 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.319859    2574 manager.go:1196] Started watching for new ooms in manager
Jul 17 09:26:41.320417 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.320405    2574 manager.go:349] Starting recovery of all containers
Jul 17 09:26:41.320961 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:41.320933    2574 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 17 09:26:41.321042 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:41.321020    2574 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 17 09:26:41.325657 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.325640    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sg5ss"
Jul 17 09:26:41.330422 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.330406    2574 manager.go:354] Recovery completed
Jul 17 09:26:41.331290 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.331274    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-sg5ss"
Jul 17 09:26:41.334599 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.334588    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 09:26:41.336230 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.336216    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 09:26:41.336360 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.336245    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 09:26:41.336360 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.336258    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-235.ec2.internal" event="NodeHasSufficientPID"
Jul 17 09:26:41.337693 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.337680    2574 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 17 09:26:41.337693 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.337692    2574 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 17 09:26:41.337794 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.337705    2574 state_mem.go:36] "Initialized new in-memory state store"
Jul 17 09:26:41.337794 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.337712    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 09:26:41.338094 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:41.338037    2574 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-235.ec2.internal.18c309924d565c9d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-235.ec2.internal,UID:ip-10-0-136-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-235.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-235.ec2.internal,},FirstTimestamp:2026-07-17 09:26:41.336229021 +0000 UTC m=+0.391811898,LastTimestamp:2026-07-17 09:26:41.336229021 +0000 UTC m=+0.391811898,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-235.ec2.internal,}"
Jul 17 09:26:41.338507 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.338490    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 09:26:41.338569 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.338519    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 09:26:41.338569 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.338530    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-235.ec2.internal" event="NodeHasSufficientPID"
Jul 17 09:26:41.338810 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.338794    2574 policy_none.go:49] "None policy: Start"
Jul 17 09:26:41.338810 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.338812    2574 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 17 09:26:41.338915 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.338821    2574 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 17 09:26:41.339926 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.339915    2574 policy_none.go:47] "Start"
Jul 17 09:26:41.379925 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.379809    2574 manager.go:337] "Starting Device Plugin manager"
Jul 17 09:26:41.381211 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:41.379945    2574 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 17 09:26:41.381211 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.379958    2574 server.go:87] "Starting device plugin registration server"
Jul 17 09:26:41.381211 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.380128    2574 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 17 09:26:41.381211 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.380136    2574 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 17 09:26:41.381211 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.380238    2574 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 17 09:26:41.381211 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.380330    2574 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 17 09:26:41.381211 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.380336    2574 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 17 09:26:41.381211 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:41.380750    2574 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 17 09:26:41.381211 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:41.380797    2574 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-235.ec2.internal\" not found"
Jul 17 09:26:41.416460 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.416440    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 17 09:26:41.417726 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.417708    2574 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 17 09:26:41.417811 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.417744    2574 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 17 09:26:41.417811 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.417763    2574 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 17 09:26:41.417811 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:41.417796    2574 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 17 09:26:41.420107 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.420084    2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 09:26:41.480786 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.480740    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 09:26:41.481462 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.481450    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 09:26:41.481528 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.481474    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 09:26:41.481528 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.481483    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-235.ec2.internal" event="NodeHasSufficientPID"
Jul 17 09:26:41.481528 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.481502    2574 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-235.ec2.internal"
Jul 17 09:26:41.489781 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.489769    2574 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-235.ec2.internal"
Jul 17 09:26:41.489824 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:41.489787    2574 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-235.ec2.internal\": node \"ip-10-0-136-235.ec2.internal\" not found"
Jul 17 09:26:41.512807 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:41.512775    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-235.ec2.internal\" not found"
Jul 17 09:26:41.518154 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.518140    2574 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-235.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-235.ec2.internal"]
Jul 17 09:26:41.518202 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.518188    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 09:26:41.518925 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.518905    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 09:26:41.519009 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.518938    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 09:26:41.519009 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.518949    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-235.ec2.internal" event="NodeHasSufficientPID"
Jul 17 09:26:41.520230 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.520218    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 09:26:41.520374 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.520361    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-235.ec2.internal"
Jul 17 09:26:41.520435 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.520394    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 09:26:41.520881 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.520863    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 09:26:41.520946 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.520891    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 09:26:41.520946 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.520899    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 09:26:41.520946 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.520906    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-235.ec2.internal" event="NodeHasSufficientPID"
Jul 17 09:26:41.520946 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.520916    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 09:26:41.520946 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.520929    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-235.ec2.internal" event="NodeHasSufficientPID"
Jul 17 09:26:41.522249 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.522234    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-235.ec2.internal"
Jul 17 09:26:41.522333 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.522258    2574 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 17 09:26:41.522874 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.522858    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 17 09:26:41.522938 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.522892    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 17 09:26:41.522938 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.522909    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-235.ec2.internal" event="NodeHasSufficientPID"
Jul 17 09:26:41.538796 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:41.538774    2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-235.ec2.internal\" not found" node="ip-10-0-136-235.ec2.internal"
Jul 17 09:26:41.543036 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:41.543021    2574 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-235.ec2.internal\" not found" node="ip-10-0-136-235.ec2.internal"
Jul 17 09:26:41.613513 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:41.613495    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-235.ec2.internal\" not found"
Jul 17 09:26:41.615584 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.615566    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b5a95e7ea41ed108b0a5006ff67827fb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-235.ec2.internal\" (UID: \"b5a95e7ea41ed108b0a5006ff67827fb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-235.ec2.internal"
Jul 17 09:26:41.615666 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.615596    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5a95e7ea41ed108b0a5006ff67827fb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-235.ec2.internal\" (UID: \"b5a95e7ea41ed108b0a5006ff67827fb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-235.ec2.internal"
Jul 17 09:26:41.615666 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.615620    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fa208b626e54278e56162a86ee4791d7-config\") pod \"kube-apiserver-proxy-ip-10-0-136-235.ec2.internal\" (UID: \"fa208b626e54278e56162a86ee4791d7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-235.ec2.internal"
Jul 17 09:26:41.713638 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:41.713615    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-235.ec2.internal\" not found"
Jul 17 09:26:41.715816 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.715804    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b5a95e7ea41ed108b0a5006ff67827fb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-235.ec2.internal\" (UID: \"b5a95e7ea41ed108b0a5006ff67827fb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-235.ec2.internal"
Jul 17 09:26:41.715866 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.715825    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5a95e7ea41ed108b0a5006ff67827fb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-235.ec2.internal\" (UID: \"b5a95e7ea41ed108b0a5006ff67827fb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-235.ec2.internal"
Jul 17 09:26:41.715866 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.715842    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fa208b626e54278e56162a86ee4791d7-config\") pod \"kube-apiserver-proxy-ip-10-0-136-235.ec2.internal\" (UID: \"fa208b626e54278e56162a86ee4791d7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-235.ec2.internal"
Jul 17 09:26:41.715954 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.715876    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fa208b626e54278e56162a86ee4791d7-config\") pod \"kube-apiserver-proxy-ip-10-0-136-235.ec2.internal\" (UID: \"fa208b626e54278e56162a86ee4791d7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-235.ec2.internal"
Jul 17 09:26:41.715954 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.715897    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5a95e7ea41ed108b0a5006ff67827fb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-235.ec2.internal\" (UID: \"b5a95e7ea41ed108b0a5006ff67827fb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-235.ec2.internal"
Jul 17 09:26:41.715954 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.715899    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b5a95e7ea41ed108b0a5006ff67827fb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-235.ec2.internal\" (UID: \"b5a95e7ea41ed108b0a5006ff67827fb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-235.ec2.internal"
Jul 17 09:26:41.814251 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:41.814206    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-235.ec2.internal\" not found"
Jul 17 09:26:41.841527 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.841513    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-235.ec2.internal"
Jul 17 09:26:41.847039 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:41.847024    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-235.ec2.internal"
Jul 17 09:26:41.914705 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:41.914684    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-235.ec2.internal\" not found"
Jul 17 09:26:42.015059 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:42.015035    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-235.ec2.internal\" not found"
Jul 17 09:26:42.115545 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:42.115500    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-235.ec2.internal\" not found"
Jul 17 09:26:42.215986 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:42.215960    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-235.ec2.internal\" not found"
Jul 17 09:26:42.250272 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:42.250254    2574 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 17 09:26:42.250615 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:42.250377    2574 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 17 09:26:42.316609 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:42.316588    2574 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-235.ec2.internal\" not found"
Jul 17 09:26:42.337207 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:42.337176    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-16 09:21:42 +0000 UTC" deadline="2028-03-08 09:42:59.767119479 +0000 UTC"
Jul 17 09:26:42.337207 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:42.337203    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14400h16m17.429919744s"
Jul 17 09:26:42.356967 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:42.356952    2574 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 09:26:42.362156 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:42.362140    2574 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 09:26:42.414357 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:42.414315    2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-235.ec2.internal"
Jul 17 09:26:42.427371 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:42.427350    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 17 09:26:42.428361 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:42.428344    2574 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-235.ec2.internal"
Jul 17 09:26:42.436568 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:42.436550    2574 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 17 09:26:42.509384 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:42.509357    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa208b626e54278e56162a86ee4791d7.slice/crio-20e3d6d14c159f6c5bcc8b4cf641c3bf3b99278b0af7d8d27e4f643c6bc79cb6 WatchSource:0}: Error finding container 20e3d6d14c159f6c5bcc8b4cf641c3bf3b99278b0af7d8d27e4f643c6bc79cb6: Status 404 returned error can't find the container with id 20e3d6d14c159f6c5bcc8b4cf641c3bf3b99278b0af7d8d27e4f643c6bc79cb6
Jul 17 09:26:42.509782 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:42.509759    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5a95e7ea41ed108b0a5006ff67827fb.slice/crio-f3495ae47af69aec0765ba693e14f26395bf7830a4a06b82e8e5f34aab17219c WatchSource:0}: Error finding container f3495ae47af69aec0765ba693e14f26395bf7830a4a06b82e8e5f34aab17219c: Status 404 returned error can't find the container with id f3495ae47af69aec0765ba693e14f26395bf7830a4a06b82e8e5f34aab17219c
Jul 17 09:26:42.514789 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:42.514770    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 09:26:42.758908 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:42.758868    2574 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 09:26:44.649400 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.649381    2574 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 17 09:26:44.764607 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.764590    2574 apiserver.go:52] "Watching apiserver"
Jul 17 09:26:44.772856 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.772835    2574 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 17 09:26:44.775203 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.775171    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-dh564","kube-system/global-pull-secret-syncer-trvgv","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25","openshift-multus/multus-additional-cni-plugins-d4ksz","openshift-multus/multus-mcbxh","openshift-multus/network-metrics-daemon-s62sx","openshift-network-diagnostics/network-check-target-qks8f","openshift-network-operator/iptables-alerter-4xwhf","kube-system/konnectivity-agent-gd59l","kube-system/kube-apiserver-proxy-ip-10-0-136-235.ec2.internal","openshift-cluster-node-tuning-operator/tuned-j96bm","openshift-dns/node-resolver-fz9bt","openshift-image-registry/node-ca-4x7kz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-235.ec2.internal"]
Jul 17 09:26:44.777533 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.777508    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4xwhf"
Jul 17 09:26:44.779381 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.779359    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4x7kz"
Jul 17 09:26:44.780403 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.779900    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 17 09:26:44.780403 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.779968    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pmkkw\""
Jul 17 09:26:44.780403 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.780019    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 17 09:26:44.780403 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.780219    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 17 09:26:44.780627 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.780586    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:26:44.780680 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:44.780645    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qks8f" podUID="ba08cebc-2cf5-4174-9698-031b279f9258"
Jul 17 09:26:44.781628 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.781609    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 17 09:26:44.781793 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.781773    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-d4ksz"
Jul 17 09:26:44.782227 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.782208    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 17 09:26:44.782227 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.782223    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 17 09:26:44.782541 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.782525    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-s4rjt\""
Jul 17 09:26:44.783092 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.783075    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.784065 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.784046    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 17 09:26:44.784479 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.784459    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:26:44.784570 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:44.784522    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62sx" podUID="fb87a089-9075-4b84-a8b6-98cfaa1d9387"
Jul 17 09:26:44.784632 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.784591    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zbtpr\""
Jul 17 09:26:44.784721 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.784704    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 17 09:26:44.784847 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.784829    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 17 09:26:44.785023 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.785009    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 17 09:26:44.785208 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.785192    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 17 09:26:44.785920 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.785901    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-6tqtg\""
Jul 17 09:26:44.785920 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.785911    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 17 09:26:44.786949 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.786426    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.787396 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.787379    2574 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 17 09:26:44.787932 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.787915    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25"
Jul 17 09:26:44.789313 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.789299    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gd59l"
Jul 17 09:26:44.789698 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.789679    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 17 09:26:44.789795 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.789761    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-hsgb5\""
Jul 17 09:26:44.789795 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.789771    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 17 09:26:44.789795 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.789777    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 17 09:26:44.790046 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.789840    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 17 09:26:44.790046 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.789910    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 17 09:26:44.790046 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.789966    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 17 09:26:44.790289 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.790225    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 17 09:26:44.790378 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.790354    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 17 09:26:44.790517 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.790498    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-48ck4\""
Jul 17 09:26:44.790577 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.790565    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:44.791097 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.791082    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 17 09:26:44.791770 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.791756    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 17 09:26:44.791955 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.791910    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-tz7hj\""
Jul 17 09:26:44.792127 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.792112    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fz9bt"
Jul 17 09:26:44.792617 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.792595    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 17 09:26:44.793061 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.793045    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 17 09:26:44.793308 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.793290    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-f88sd\""
Jul 17 09:26:44.793366 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.793323    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 17 09:26:44.793366 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.793289    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:26:44.793437 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:44.793406    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trvgv" podUID="f357001d-cac3-4c7d-b427-7cc231e3d4ff"
Jul 17 09:26:44.794541 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.794525    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-qs7jz\""
Jul 17 09:26:44.794610 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.794599    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 17 09:26:44.794677 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.794641    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 17 09:26:44.797913 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.797897    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/270ef556-fd73-4a5b-bf9a-73ecab7842ca-iptables-alerter-script\") pod \"iptables-alerter-4xwhf\" (UID: \"270ef556-fd73-4a5b-bf9a-73ecab7842ca\") " pod="openshift-network-operator/iptables-alerter-4xwhf"
Jul 17 09:26:44.798000 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.797925    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-cnibin\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.798000 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.797957    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-etc-kubernetes\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.798000 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.797979    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-node-log\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.798109 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798004    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr9t7\" (UniqueName: \"kubernetes.io/projected/19ea8f77-c130-4773-895f-d2ee2c224f10-kube-api-access-jr9t7\") pod \"aws-ebs-csi-driver-node-svq25\" (UID: \"19ea8f77-c130-4773-895f-d2ee2c224f10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25"
Jul 17 09:26:44.798109 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798054    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5c165f52-605b-48e9-9cff-bf1e0b7b97af-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d4ksz\" (UID: \"5c165f52-605b-48e9-9cff-bf1e0b7b97af\") " pod="openshift-multus/multus-additional-cni-plugins-d4ksz"
Jul 17 09:26:44.798109 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798092    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-multus-socket-dir-parent\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.798196 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798122    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-host-slash\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.798196 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798147    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-run-systemd\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.798196 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798170    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-var-lib-openvswitch\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.798313 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798202    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/abec7743-4f1a-45ad-bddf-3f9c0b7d42dc-host\") pod \"node-ca-4x7kz\" (UID: \"abec7743-4f1a-45ad-bddf-3f9c0b7d42dc\") " pod="openshift-image-registry/node-ca-4x7kz"
Jul 17 09:26:44.798313 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798224    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-os-release\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.798313 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798265    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5c165f52-605b-48e9-9cff-bf1e0b7b97af-cnibin\") pod \"multus-additional-cni-plugins-d4ksz\" (UID: \"5c165f52-605b-48e9-9cff-bf1e0b7b97af\") " pod="openshift-multus/multus-additional-cni-plugins-d4ksz"
Jul 17 09:26:44.798313 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798289    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5c165f52-605b-48e9-9cff-bf1e0b7b97af-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d4ksz\" (UID: \"5c165f52-605b-48e9-9cff-bf1e0b7b97af\") " pod="openshift-multus/multus-additional-cni-plugins-d4ksz"
Jul 17 09:26:44.798471 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798320    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-host-run-k8s-cni-cncf-io\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.798471 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798344    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-host-var-lib-cni-multus\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.798471 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798367    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-host-var-lib-kubelet\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.798471 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798391    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.798471 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798413    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/84db06ab-87e2-43b3-82d0-13f06ba86d06-ovnkube-config\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.798471 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798436    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/19ea8f77-c130-4773-895f-d2ee2c224f10-etc-selinux\") pod \"aws-ebs-csi-driver-node-svq25\" (UID: \"19ea8f77-c130-4773-895f-d2ee2c224f10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25"
Jul 17 09:26:44.798471 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798457    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-system-cni-dir\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.798704 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798478    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/87fc473b-6f15-4afc-8632-62937f882dbc-multus-daemon-config\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.798704 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798507    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-host-cni-bin\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.798704 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798529    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/19ea8f77-c130-4773-895f-d2ee2c224f10-kubelet-dir\") pod \"aws-ebs-csi-driver-node-svq25\" (UID: \"19ea8f77-c130-4773-895f-d2ee2c224f10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25"
Jul 17 09:26:44.798704 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798550    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/84db06ab-87e2-43b3-82d0-13f06ba86d06-ovn-node-metrics-cert\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.798704 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798589    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5c165f52-605b-48e9-9cff-bf1e0b7b97af-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d4ksz\" (UID: \"5c165f52-605b-48e9-9cff-bf1e0b7b97af\") " pod="openshift-multus/multus-additional-cni-plugins-d4ksz"
Jul 17 09:26:44.798704 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798612    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqvlb\" (UniqueName: \"kubernetes.io/projected/5c165f52-605b-48e9-9cff-bf1e0b7b97af-kube-api-access-rqvlb\") pod \"multus-additional-cni-plugins-d4ksz\" (UID: \"5c165f52-605b-48e9-9cff-bf1e0b7b97af\") " pod="openshift-multus/multus-additional-cni-plugins-d4ksz"
Jul 17 09:26:44.798704 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798637    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/87fc473b-6f15-4afc-8632-62937f882dbc-cni-binary-copy\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.798704 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798657    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-host-var-lib-cni-bin\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.798704 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798672    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-multus-conf-dir\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.798704 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798685    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-log-socket\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.798704 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798701    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-host-cni-netd\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.799123 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798733    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-host-run-netns\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.799123 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798768    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-host-run-multus-certs\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.799123 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798803    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-run-openvswitch\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.799123 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798831    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/270ef556-fd73-4a5b-bf9a-73ecab7842ca-host-slash\") pod \"iptables-alerter-4xwhf\" (UID: \"270ef556-fd73-4a5b-bf9a-73ecab7842ca\") " pod="openshift-network-operator/iptables-alerter-4xwhf"
Jul 17 09:26:44.799123 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798860    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb87a089-9075-4b84-a8b6-98cfaa1d9387-metrics-certs\") pod \"network-metrics-daemon-s62sx\" (UID: \"fb87a089-9075-4b84-a8b6-98cfaa1d9387\") " pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:26:44.799123 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798887    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-run-ovn\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.799123 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798917    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/abec7743-4f1a-45ad-bddf-3f9c0b7d42dc-serviceca\") pod \"node-ca-4x7kz\" (UID: \"abec7743-4f1a-45ad-bddf-3f9c0b7d42dc\") " pod="openshift-image-registry/node-ca-4x7kz"
Jul 17 09:26:44.799123 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798953    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-multus-cni-dir\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.799123 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798976    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4nmr\" (UniqueName: \"kubernetes.io/projected/270ef556-fd73-4a5b-bf9a-73ecab7842ca-kube-api-access-r4nmr\") pod \"iptables-alerter-4xwhf\" (UID: \"270ef556-fd73-4a5b-bf9a-73ecab7842ca\") " pod="openshift-network-operator/iptables-alerter-4xwhf"
Jul 17 09:26:44.799123 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.798999    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c165f52-605b-48e9-9cff-bf1e0b7b97af-system-cni-dir\") pod \"multus-additional-cni-plugins-d4ksz\" (UID: \"5c165f52-605b-48e9-9cff-bf1e0b7b97af\") " pod="openshift-multus/multus-additional-cni-plugins-d4ksz"
Jul 17 09:26:44.799123 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.799022    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-host-kubelet\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.799123 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.799061    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-systemd-units\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.799123 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.799085    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-etc-openvswitch\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.799123 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.799108    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-host-run-ovn-kubernetes\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.799671 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.799133    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/84db06ab-87e2-43b3-82d0-13f06ba86d06-ovnkube-script-lib\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.799671 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.799159    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t48pc\" (UniqueName: \"kubernetes.io/projected/84db06ab-87e2-43b3-82d0-13f06ba86d06-kube-api-access-t48pc\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.799671 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.799183    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/19ea8f77-c130-4773-895f-d2ee2c224f10-device-dir\") pod \"aws-ebs-csi-driver-node-svq25\" (UID: \"19ea8f77-c130-4773-895f-d2ee2c224f10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25"
Jul 17 09:26:44.799671 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.799208    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5c165f52-605b-48e9-9cff-bf1e0b7b97af-cni-binary-copy\") pod \"multus-additional-cni-plugins-d4ksz\" (UID: \"5c165f52-605b-48e9-9cff-bf1e0b7b97af\") " pod="openshift-multus/multus-additional-cni-plugins-d4ksz"
Jul 17 09:26:44.799671 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.799232    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/84db06ab-87e2-43b3-82d0-13f06ba86d06-env-overrides\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.799671 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.799276    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58gs6\" (UniqueName: \"kubernetes.io/projected/abec7743-4f1a-45ad-bddf-3f9c0b7d42dc-kube-api-access-58gs6\") pod \"node-ca-4x7kz\" (UID: \"abec7743-4f1a-45ad-bddf-3f9c0b7d42dc\") " pod="openshift-image-registry/node-ca-4x7kz"
Jul 17 09:26:44.799671 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.799305    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf448\" (UniqueName: \"kubernetes.io/projected/ba08cebc-2cf5-4174-9698-031b279f9258-kube-api-access-cf448\") pod \"network-check-target-qks8f\" (UID: \"ba08cebc-2cf5-4174-9698-031b279f9258\") " pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:26:44.799671 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.799329    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-hostroot\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.799671 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.799351    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw4q2\" (UniqueName: \"kubernetes.io/projected/fb87a089-9075-4b84-a8b6-98cfaa1d9387-kube-api-access-nw4q2\") pod \"network-metrics-daemon-s62sx\" (UID: \"fb87a089-9075-4b84-a8b6-98cfaa1d9387\") " pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:26:44.799671 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.799368    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/19ea8f77-c130-4773-895f-d2ee2c224f10-socket-dir\") pod \"aws-ebs-csi-driver-node-svq25\" (UID: \"19ea8f77-c130-4773-895f-d2ee2c224f10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25"
Jul 17 09:26:44.799671 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.799390    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/19ea8f77-c130-4773-895f-d2ee2c224f10-registration-dir\") pod \"aws-ebs-csi-driver-node-svq25\" (UID: \"19ea8f77-c130-4773-895f-d2ee2c224f10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25"
Jul 17 09:26:44.799671 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.799422    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5c165f52-605b-48e9-9cff-bf1e0b7b97af-os-release\") pod \"multus-additional-cni-plugins-d4ksz\" (UID: \"5c165f52-605b-48e9-9cff-bf1e0b7b97af\") " pod="openshift-multus/multus-additional-cni-plugins-d4ksz"
Jul 17 09:26:44.799671 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.799458    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c655k\" (UniqueName: \"kubernetes.io/projected/87fc473b-6f15-4afc-8632-62937f882dbc-kube-api-access-c655k\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.799671 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.799484    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/19ea8f77-c130-4773-895f-d2ee2c224f10-sys-fs\") pod \"aws-ebs-csi-driver-node-svq25\" (UID: \"19ea8f77-c130-4773-895f-d2ee2c224f10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25"
Jul 17 09:26:44.799671 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.799507    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-host-run-netns\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.802408 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.802391    2574 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 17 09:26:44.889023 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.888958    2574 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 17 09:26:44.895083 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.895038    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-235.ec2.internal" event={"ID":"fa208b626e54278e56162a86ee4791d7","Type":"ContainerStarted","Data":"20e3d6d14c159f6c5bcc8b4cf641c3bf3b99278b0af7d8d27e4f643c6bc79cb6"}
Jul 17 09:26:44.896150 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.896120    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-235.ec2.internal" event={"ID":"b5a95e7ea41ed108b0a5006ff67827fb","Type":"ContainerStarted","Data":"f3495ae47af69aec0765ba693e14f26395bf7830a4a06b82e8e5f34aab17219c"}
Jul 17 09:26:44.900428 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.900400    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f357001d-cac3-4c7d-b427-7cc231e3d4ff-original-pull-secret\") pod \"global-pull-secret-syncer-trvgv\" (UID: \"f357001d-cac3-4c7d-b427-7cc231e3d4ff\") " pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:26:44.900540 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.900451    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-multus-socket-dir-parent\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.900540 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.900487    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-host-slash\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.900540 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.900510    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-run-systemd\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.900540 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.900534    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-var-lib-openvswitch\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.900741 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.900554    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/abec7743-4f1a-45ad-bddf-3f9c0b7d42dc-host\") pod \"node-ca-4x7kz\" (UID: \"abec7743-4f1a-45ad-bddf-3f9c0b7d42dc\") " pod="openshift-image-registry/node-ca-4x7kz"
Jul 17 09:26:44.900741 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.900572    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-multus-socket-dir-parent\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.900741 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.900627    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-host-slash\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.900741 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.900629    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-run-systemd\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.900741 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.900578    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-etc-sysconfig\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:44.900741 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.900661    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-var-lib-openvswitch\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.900741 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.900664    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-os-release\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.900741 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.900701    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5c165f52-605b-48e9-9cff-bf1e0b7b97af-cnibin\") pod \"multus-additional-cni-plugins-d4ksz\" (UID: \"5c165f52-605b-48e9-9cff-bf1e0b7b97af\") " pod="openshift-multus/multus-additional-cni-plugins-d4ksz"
Jul 17 09:26:44.900741 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.900709    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-os-release\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.900741 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.900726    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5c165f52-605b-48e9-9cff-bf1e0b7b97af-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d4ksz\" (UID: \"5c165f52-605b-48e9-9cff-bf1e0b7b97af\") " pod="openshift-multus/multus-additional-cni-plugins-d4ksz"
Jul 17 09:26:44.901212 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.900752    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/abec7743-4f1a-45ad-bddf-3f9c0b7d42dc-host\") pod \"node-ca-4x7kz\" (UID: \"abec7743-4f1a-45ad-bddf-3f9c0b7d42dc\") " pod="openshift-image-registry/node-ca-4x7kz"
Jul 17 09:26:44.901212 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.900753    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-var-lib-kubelet\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:44.901212 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.900778    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5c165f52-605b-48e9-9cff-bf1e0b7b97af-cnibin\") pod \"multus-additional-cni-plugins-d4ksz\" (UID: \"5c165f52-605b-48e9-9cff-bf1e0b7b97af\") " pod="openshift-multus/multus-additional-cni-plugins-d4ksz"
Jul 17 09:26:44.901212 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.900778    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4qcj\" (UniqueName: \"kubernetes.io/projected/eccc078f-2e4c-4a45-9056-d0cd7053faf6-kube-api-access-d4qcj\") pod \"node-resolver-fz9bt\" (UID: \"eccc078f-2e4c-4a45-9056-d0cd7053faf6\") " pod="openshift-dns/node-resolver-fz9bt"
Jul 17 09:26:44.901212 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.900816    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-host-run-k8s-cni-cncf-io\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.901212 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.900840    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-host-var-lib-cni-multus\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.901212 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.900862    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-host-var-lib-kubelet\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.901212 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.900891    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.901212 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.900918    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/84db06ab-87e2-43b3-82d0-13f06ba86d06-ovnkube-config\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.901212 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.900942    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/19ea8f77-c130-4773-895f-d2ee2c224f10-etc-selinux\") pod \"aws-ebs-csi-driver-node-svq25\" (UID: \"19ea8f77-c130-4773-895f-d2ee2c224f10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25"
Jul 17 09:26:44.901212 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.900967    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-etc-sysctl-conf\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:44.901212 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.900991    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eccc078f-2e4c-4a45-9056-d0cd7053faf6-tmp-dir\") pod \"node-resolver-fz9bt\" (UID: \"eccc078f-2e4c-4a45-9056-d0cd7053faf6\") " pod="openshift-dns/node-resolver-fz9bt"
Jul 17 09:26:44.901212 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901016    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-system-cni-dir\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.901212 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901039    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/87fc473b-6f15-4afc-8632-62937f882dbc-multus-daemon-config\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.901212 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901064    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-host-cni-bin\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.901212 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901087    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/19ea8f77-c130-4773-895f-d2ee2c224f10-kubelet-dir\") pod \"aws-ebs-csi-driver-node-svq25\" (UID: \"19ea8f77-c130-4773-895f-d2ee2c224f10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25"
Jul 17 09:26:44.901212 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901119    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-etc-systemd\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:44.902004 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901161    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f357001d-cac3-4c7d-b427-7cc231e3d4ff-kubelet-config\") pod \"global-pull-secret-syncer-trvgv\" (UID: \"f357001d-cac3-4c7d-b427-7cc231e3d4ff\") " pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:26:44.902004 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901190    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/84db06ab-87e2-43b3-82d0-13f06ba86d06-ovn-node-metrics-cert\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.902004 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901215    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5c165f52-605b-48e9-9cff-bf1e0b7b97af-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d4ksz\" (UID: \"5c165f52-605b-48e9-9cff-bf1e0b7b97af\") " pod="openshift-multus/multus-additional-cni-plugins-d4ksz"
Jul 17 09:26:44.902004 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901262    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqvlb\" (UniqueName: \"kubernetes.io/projected/5c165f52-605b-48e9-9cff-bf1e0b7b97af-kube-api-access-rqvlb\") pod \"multus-additional-cni-plugins-d4ksz\" (UID: \"5c165f52-605b-48e9-9cff-bf1e0b7b97af\") " pod="openshift-multus/multus-additional-cni-plugins-d4ksz"
Jul 17 09:26:44.902004 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901265    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5c165f52-605b-48e9-9cff-bf1e0b7b97af-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-d4ksz\" (UID: \"5c165f52-605b-48e9-9cff-bf1e0b7b97af\") " pod="openshift-multus/multus-additional-cni-plugins-d4ksz"
Jul 17 09:26:44.902004 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901285    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2206a53-85f4-42a1-bd3b-2bbd84db2978-tmp\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:44.902004 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901327    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/87fc473b-6f15-4afc-8632-62937f882dbc-cni-binary-copy\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.902004 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901354    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-host-var-lib-cni-bin\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.902004 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901378    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-multus-conf-dir\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.902004 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901402    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-log-socket\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.902004 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901426    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-host-cni-netd\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.902004 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901427    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-host-var-lib-kubelet\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.902004 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901433    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-system-cni-dir\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.902004 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901450    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-host-run-netns\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.902004 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901476    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-host-run-multus-certs\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.902004 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901489    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-host-var-lib-cni-multus\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.902004 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901486    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.902827 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901503    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-run-openvswitch\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.902827 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901518    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-host-run-k8s-cni-cncf-io\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.902827 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901538    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/270ef556-fd73-4a5b-bf9a-73ecab7842ca-host-slash\") pod \"iptables-alerter-4xwhf\" (UID: \"270ef556-fd73-4a5b-bf9a-73ecab7842ca\") " pod="openshift-network-operator/iptables-alerter-4xwhf"
Jul 17 09:26:44.902827 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901547    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-run-openvswitch\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.902827 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901568    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-etc-kubernetes\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:44.902827 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901591    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f357001d-cac3-4c7d-b427-7cc231e3d4ff-dbus\") pod \"global-pull-secret-syncer-trvgv\" (UID: \"f357001d-cac3-4c7d-b427-7cc231e3d4ff\") " pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:26:44.902827 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901614    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb87a089-9075-4b84-a8b6-98cfaa1d9387-metrics-certs\") pod \"network-metrics-daemon-s62sx\" (UID: \"fb87a089-9075-4b84-a8b6-98cfaa1d9387\") " pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:26:44.902827 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901640    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-run-ovn\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.902827 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901642    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/19ea8f77-c130-4773-895f-d2ee2c224f10-etc-selinux\") pod \"aws-ebs-csi-driver-node-svq25\" (UID: \"19ea8f77-c130-4773-895f-d2ee2c224f10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25"
Jul 17 09:26:44.902827 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901673    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/abec7743-4f1a-45ad-bddf-3f9c0b7d42dc-serviceca\") pod \"node-ca-4x7kz\" (UID: \"abec7743-4f1a-45ad-bddf-3f9c0b7d42dc\") " pod="openshift-image-registry/node-ca-4x7kz"
Jul 17 09:26:44.902827 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901688    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/270ef556-fd73-4a5b-bf9a-73ecab7842ca-host-slash\") pod \"iptables-alerter-4xwhf\" (UID: \"270ef556-fd73-4a5b-bf9a-73ecab7842ca\") " pod="openshift-network-operator/iptables-alerter-4xwhf"
Jul 17 09:26:44.902827 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901701    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-etc-sysctl-d\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:44.902827 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901745    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-multus-cni-dir\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.902827 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901772    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4nmr\" (UniqueName: \"kubernetes.io/projected/270ef556-fd73-4a5b-bf9a-73ecab7842ca-kube-api-access-r4nmr\") pod \"iptables-alerter-4xwhf\" (UID: \"270ef556-fd73-4a5b-bf9a-73ecab7842ca\") " pod="openshift-network-operator/iptables-alerter-4xwhf"
Jul 17 09:26:44.902827 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901797    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c165f52-605b-48e9-9cff-bf1e0b7b97af-system-cni-dir\") pod \"multus-additional-cni-plugins-d4ksz\" (UID: \"5c165f52-605b-48e9-9cff-bf1e0b7b97af\") " pod="openshift-multus/multus-additional-cni-plugins-d4ksz"
Jul 17 09:26:44.902827 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901822    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-etc-modprobe-d\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:44.902827 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901852    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-host-kubelet\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.903598 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901878    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-systemd-units\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.903598 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901904    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-etc-openvswitch\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.903598 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901932    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-host-run-ovn-kubernetes\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.903598 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901961    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/84db06ab-87e2-43b3-82d0-13f06ba86d06-ovnkube-script-lib\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.903598 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.901989    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t48pc\" (UniqueName: \"kubernetes.io/projected/84db06ab-87e2-43b3-82d0-13f06ba86d06-kube-api-access-t48pc\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.903598 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902032    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/19ea8f77-c130-4773-895f-d2ee2c224f10-device-dir\") pod \"aws-ebs-csi-driver-node-svq25\" (UID: \"19ea8f77-c130-4773-895f-d2ee2c224f10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25"
Jul 17 09:26:44.903598 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902061    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5c165f52-605b-48e9-9cff-bf1e0b7b97af-cni-binary-copy\") pod \"multus-additional-cni-plugins-d4ksz\" (UID: \"5c165f52-605b-48e9-9cff-bf1e0b7b97af\") " pod="openshift-multus/multus-additional-cni-plugins-d4ksz"
Jul 17 09:26:44.903598 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902131    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/84db06ab-87e2-43b3-82d0-13f06ba86d06-env-overrides\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.903598 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902159    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58gs6\" (UniqueName: \"kubernetes.io/projected/abec7743-4f1a-45ad-bddf-3f9c0b7d42dc-kube-api-access-58gs6\") pod \"node-ca-4x7kz\" (UID: \"abec7743-4f1a-45ad-bddf-3f9c0b7d42dc\") " pod="openshift-image-registry/node-ca-4x7kz"
Jul 17 09:26:44.903598 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902188    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf448\" (UniqueName: \"kubernetes.io/projected/ba08cebc-2cf5-4174-9698-031b279f9258-kube-api-access-cf448\") pod \"network-check-target-qks8f\" (UID: \"ba08cebc-2cf5-4174-9698-031b279f9258\") " pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:26:44.903598 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902196    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/87fc473b-6f15-4afc-8632-62937f882dbc-multus-daemon-config\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.903598 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902212    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/84db06ab-87e2-43b3-82d0-13f06ba86d06-ovnkube-config\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.903598 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902216    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-lib-modules\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:44.903598 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902260    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eccc078f-2e4c-4a45-9056-d0cd7053faf6-hosts-file\") pod \"node-resolver-fz9bt\" (UID: \"eccc078f-2e4c-4a45-9056-d0cd7053faf6\") " pod="openshift-dns/node-resolver-fz9bt"
Jul 17 09:26:44.903598 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902282    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-host-cni-bin\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.903598 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902288    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-hostroot\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.903598 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902315    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw4q2\" (UniqueName: \"kubernetes.io/projected/fb87a089-9075-4b84-a8b6-98cfaa1d9387-kube-api-access-nw4q2\") pod \"network-metrics-daemon-s62sx\" (UID: \"fb87a089-9075-4b84-a8b6-98cfaa1d9387\") " pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:26:44.904413 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902339    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/19ea8f77-c130-4773-895f-d2ee2c224f10-kubelet-dir\") pod \"aws-ebs-csi-driver-node-svq25\" (UID: \"19ea8f77-c130-4773-895f-d2ee2c224f10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25"
Jul 17 09:26:44.904413 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:44.902346    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:26:44.904413 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902339    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/19ea8f77-c130-4773-895f-d2ee2c224f10-socket-dir\") pod \"aws-ebs-csi-driver-node-svq25\" (UID: \"19ea8f77-c130-4773-895f-d2ee2c224f10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25"
Jul 17 09:26:44.904413 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902401    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/19ea8f77-c130-4773-895f-d2ee2c224f10-registration-dir\") pod \"aws-ebs-csi-driver-node-svq25\" (UID: \"19ea8f77-c130-4773-895f-d2ee2c224f10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25"
Jul 17 09:26:44.904413 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902428    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5c165f52-605b-48e9-9cff-bf1e0b7b97af-os-release\") pod \"multus-additional-cni-plugins-d4ksz\" (UID: \"5c165f52-605b-48e9-9cff-bf1e0b7b97af\") " pod="openshift-multus/multus-additional-cni-plugins-d4ksz"
Jul 17 09:26:44.904413 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902434    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/19ea8f77-c130-4773-895f-d2ee2c224f10-socket-dir\") pod \"aws-ebs-csi-driver-node-svq25\" (UID: \"19ea8f77-c130-4773-895f-d2ee2c224f10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25"
Jul 17 09:26:44.904413 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902456    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0f1dd372-9aca-4b16-8020-eee6e421e6a2-konnectivity-ca\") pod \"konnectivity-agent-gd59l\" (UID: \"0f1dd372-9aca-4b16-8020-eee6e421e6a2\") " pod="kube-system/konnectivity-agent-gd59l"
Jul 17 09:26:44.904413 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902481    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-run\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:44.904413 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902505    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-host\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:44.904413 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902529    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgr7r\" (UniqueName: \"kubernetes.io/projected/f2206a53-85f4-42a1-bd3b-2bbd84db2978-kube-api-access-qgr7r\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:44.904413 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902556    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c655k\" (UniqueName: \"kubernetes.io/projected/87fc473b-6f15-4afc-8632-62937f882dbc-kube-api-access-c655k\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.904413 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902583    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/19ea8f77-c130-4773-895f-d2ee2c224f10-sys-fs\") pod \"aws-ebs-csi-driver-node-svq25\" (UID: \"19ea8f77-c130-4773-895f-d2ee2c224f10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25"
Jul 17 09:26:44.904413 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902586    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-run-ovn\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.904413 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902608    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0f1dd372-9aca-4b16-8020-eee6e421e6a2-agent-certs\") pod \"konnectivity-agent-gd59l\" (UID: \"0f1dd372-9aca-4b16-8020-eee6e421e6a2\") " pod="kube-system/konnectivity-agent-gd59l"
Jul 17 09:26:44.904413 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902632    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-sys\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:44.904413 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902656    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f2206a53-85f4-42a1-bd3b-2bbd84db2978-etc-tuned\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:44.904413 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902687    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-host-run-netns\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.905208 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902719    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/270ef556-fd73-4a5b-bf9a-73ecab7842ca-iptables-alerter-script\") pod \"iptables-alerter-4xwhf\" (UID: \"270ef556-fd73-4a5b-bf9a-73ecab7842ca\") " pod="openshift-network-operator/iptables-alerter-4xwhf"
Jul 17 09:26:44.905208 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902754    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-cnibin\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.905208 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902779    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-etc-kubernetes\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.905208 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902834    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-node-log\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.905208 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902856    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/abec7743-4f1a-45ad-bddf-3f9c0b7d42dc-serviceca\") pod \"node-ca-4x7kz\" (UID: \"abec7743-4f1a-45ad-bddf-3f9c0b7d42dc\") " pod="openshift-image-registry/node-ca-4x7kz"
Jul 17 09:26:44.905208 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902884    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr9t7\" (UniqueName: \"kubernetes.io/projected/19ea8f77-c130-4773-895f-d2ee2c224f10-kube-api-access-jr9t7\") pod \"aws-ebs-csi-driver-node-svq25\" (UID: \"19ea8f77-c130-4773-895f-d2ee2c224f10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25"
Jul 17 09:26:44.905208 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902911    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5c165f52-605b-48e9-9cff-bf1e0b7b97af-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d4ksz\" (UID: \"5c165f52-605b-48e9-9cff-bf1e0b7b97af\") " pod="openshift-multus/multus-additional-cni-plugins-d4ksz"
Jul 17 09:26:44.905208 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902937    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5c165f52-605b-48e9-9cff-bf1e0b7b97af-os-release\") pod \"multus-additional-cni-plugins-d4ksz\" (UID: \"5c165f52-605b-48e9-9cff-bf1e0b7b97af\") " pod="openshift-multus/multus-additional-cni-plugins-d4ksz"
Jul 17 09:26:44.905208 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902982    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-host-kubelet\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.905208 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.903031    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-systemd-units\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.905208 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.903079    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-etc-openvswitch\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.905208 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.903079    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5c165f52-605b-48e9-9cff-bf1e0b7b97af-tuning-conf-dir\") pod \"multus-additional-cni-plugins-d4ksz\" (UID: \"5c165f52-605b-48e9-9cff-bf1e0b7b97af\") " pod="openshift-multus/multus-additional-cni-plugins-d4ksz"
Jul 17 09:26:44.905208 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.903123    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-host-run-ovn-kubernetes\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.905208 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.903269    2574 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 17 09:26:44.905208 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.903584    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/19ea8f77-c130-4773-895f-d2ee2c224f10-device-dir\") pod \"aws-ebs-csi-driver-node-svq25\" (UID: \"19ea8f77-c130-4773-895f-d2ee2c224f10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25"
Jul 17 09:26:44.905208 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.903611    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/84db06ab-87e2-43b3-82d0-13f06ba86d06-ovnkube-script-lib\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.905208 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.903635    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/19ea8f77-c130-4773-895f-d2ee2c224f10-sys-fs\") pod \"aws-ebs-csi-driver-node-svq25\" (UID: \"19ea8f77-c130-4773-895f-d2ee2c224f10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25"
Jul 17 09:26:44.905208 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.903741    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-host-run-netns\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.905759 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.903863    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-hostroot\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.905759 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.904026    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5c165f52-605b-48e9-9cff-bf1e0b7b97af-cni-binary-copy\") pod \"multus-additional-cni-plugins-d4ksz\" (UID: \"5c165f52-605b-48e9-9cff-bf1e0b7b97af\") " pod="openshift-multus/multus-additional-cni-plugins-d4ksz"
Jul 17 09:26:44.905759 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.904357    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/270ef556-fd73-4a5b-bf9a-73ecab7842ca-iptables-alerter-script\") pod \"iptables-alerter-4xwhf\" (UID: \"270ef556-fd73-4a5b-bf9a-73ecab7842ca\") " pod="openshift-network-operator/iptables-alerter-4xwhf"
Jul 17 09:26:44.905759 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.904405    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/84db06ab-87e2-43b3-82d0-13f06ba86d06-env-overrides\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.905759 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.904424    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-cnibin\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.905759 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.904434    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-log-socket\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.905759 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.904466    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/19ea8f77-c130-4773-895f-d2ee2c224f10-registration-dir\") pod \"aws-ebs-csi-driver-node-svq25\" (UID: \"19ea8f77-c130-4773-895f-d2ee2c224f10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25"
Jul 17 09:26:44.905759 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.904481    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-node-log\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.905759 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.904511    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/84db06ab-87e2-43b3-82d0-13f06ba86d06-host-cni-netd\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.905759 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.904563    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-host-run-netns\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.905759 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.904607    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-host-run-multus-certs\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.905759 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.904686    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-multus-conf-dir\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.905759 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.904758    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-host-var-lib-cni-bin\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.905759 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.904795    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-etc-kubernetes\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.905759 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.904970    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5c165f52-605b-48e9-9cff-bf1e0b7b97af-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-d4ksz\" (UID: \"5c165f52-605b-48e9-9cff-bf1e0b7b97af\") " pod="openshift-multus/multus-additional-cni-plugins-d4ksz"
Jul 17 09:26:44.905759 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.902935    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c165f52-605b-48e9-9cff-bf1e0b7b97af-system-cni-dir\") pod \"multus-additional-cni-plugins-d4ksz\" (UID: \"5c165f52-605b-48e9-9cff-bf1e0b7b97af\") " pod="openshift-multus/multus-additional-cni-plugins-d4ksz"
Jul 17 09:26:44.905759 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.905009    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/87fc473b-6f15-4afc-8632-62937f882dbc-cni-binary-copy\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.906371 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:44.905037    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb87a089-9075-4b84-a8b6-98cfaa1d9387-metrics-certs podName:fb87a089-9075-4b84-a8b6-98cfaa1d9387 nodeName:}" failed. No retries permitted until 2026-07-17 09:26:45.405018252 +0000 UTC m=+2.987656610 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fb87a089-9075-4b84-a8b6-98cfaa1d9387-metrics-certs") pod "network-metrics-daemon-s62sx" (UID: "fb87a089-9075-4b84-a8b6-98cfaa1d9387") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:26:44.906371 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.905082    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/87fc473b-6f15-4afc-8632-62937f882dbc-multus-cni-dir\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.906486 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.906475    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/84db06ab-87e2-43b3-82d0-13f06ba86d06-ovn-node-metrics-cert\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.909855 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.909836    2574 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-zvf52"
Jul 17 09:26:44.909943 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:44.909898    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 09:26:44.909943 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:44.909919    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 09:26:44.909943 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:44.909932    2574 projected.go:196] Error preparing data for projected volume kube-api-access-cf448 for pod openshift-network-diagnostics/network-check-target-qks8f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:26:44.910083 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:44.909988    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba08cebc-2cf5-4174-9698-031b279f9258-kube-api-access-cf448 podName:ba08cebc-2cf5-4174-9698-031b279f9258 nodeName:}" failed. No retries permitted until 2026-07-17 09:26:45.409972102 +0000 UTC m=+2.992610477 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-cf448" (UniqueName: "kubernetes.io/projected/ba08cebc-2cf5-4174-9698-031b279f9258-kube-api-access-cf448") pod "network-check-target-qks8f" (UID: "ba08cebc-2cf5-4174-9698-031b279f9258") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:26:44.912136 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.912113    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4nmr\" (UniqueName: \"kubernetes.io/projected/270ef556-fd73-4a5b-bf9a-73ecab7842ca-kube-api-access-r4nmr\") pod \"iptables-alerter-4xwhf\" (UID: \"270ef556-fd73-4a5b-bf9a-73ecab7842ca\") " pod="openshift-network-operator/iptables-alerter-4xwhf"
Jul 17 09:26:44.912271 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.912150    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c655k\" (UniqueName: \"kubernetes.io/projected/87fc473b-6f15-4afc-8632-62937f882dbc-kube-api-access-c655k\") pod \"multus-mcbxh\" (UID: \"87fc473b-6f15-4afc-8632-62937f882dbc\") " pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:44.912680 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.912656    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw4q2\" (UniqueName: \"kubernetes.io/projected/fb87a089-9075-4b84-a8b6-98cfaa1d9387-kube-api-access-nw4q2\") pod \"network-metrics-daemon-s62sx\" (UID: \"fb87a089-9075-4b84-a8b6-98cfaa1d9387\") " pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:26:44.912916 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.912849    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t48pc\" (UniqueName: \"kubernetes.io/projected/84db06ab-87e2-43b3-82d0-13f06ba86d06-kube-api-access-t48pc\") pod \"ovnkube-node-dh564\" (UID: \"84db06ab-87e2-43b3-82d0-13f06ba86d06\") " pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:44.913317 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.913221    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58gs6\" (UniqueName: \"kubernetes.io/projected/abec7743-4f1a-45ad-bddf-3f9c0b7d42dc-kube-api-access-58gs6\") pod \"node-ca-4x7kz\" (UID: \"abec7743-4f1a-45ad-bddf-3f9c0b7d42dc\") " pod="openshift-image-registry/node-ca-4x7kz"
Jul 17 09:26:44.913741 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.913713    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr9t7\" (UniqueName: \"kubernetes.io/projected/19ea8f77-c130-4773-895f-d2ee2c224f10-kube-api-access-jr9t7\") pod \"aws-ebs-csi-driver-node-svq25\" (UID: \"19ea8f77-c130-4773-895f-d2ee2c224f10\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25"
Jul 17 09:26:44.914359 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.914340    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqvlb\" (UniqueName: \"kubernetes.io/projected/5c165f52-605b-48e9-9cff-bf1e0b7b97af-kube-api-access-rqvlb\") pod \"multus-additional-cni-plugins-d4ksz\" (UID: \"5c165f52-605b-48e9-9cff-bf1e0b7b97af\") " pod="openshift-multus/multus-additional-cni-plugins-d4ksz"
Jul 17 09:26:44.918914 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:44.918898    2574 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-zvf52"
Jul 17 09:26:45.004750 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.003477    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-lib-modules\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.004750 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.003513    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eccc078f-2e4c-4a45-9056-d0cd7053faf6-hosts-file\") pod \"node-resolver-fz9bt\" (UID: \"eccc078f-2e4c-4a45-9056-d0cd7053faf6\") " pod="openshift-dns/node-resolver-fz9bt"
Jul 17 09:26:45.004750 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.003537    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0f1dd372-9aca-4b16-8020-eee6e421e6a2-konnectivity-ca\") pod \"konnectivity-agent-gd59l\" (UID: \"0f1dd372-9aca-4b16-8020-eee6e421e6a2\") " pod="kube-system/konnectivity-agent-gd59l"
Jul 17 09:26:45.004750 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.003572    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-run\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.004750 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.003595    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-host\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.004750 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.003614    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgr7r\" (UniqueName: \"kubernetes.io/projected/f2206a53-85f4-42a1-bd3b-2bbd84db2978-kube-api-access-qgr7r\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.004750 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.003634    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0f1dd372-9aca-4b16-8020-eee6e421e6a2-agent-certs\") pod \"konnectivity-agent-gd59l\" (UID: \"0f1dd372-9aca-4b16-8020-eee6e421e6a2\") " pod="kube-system/konnectivity-agent-gd59l"
Jul 17 09:26:45.004750 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.003647    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-sys\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.004750 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.003664    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-lib-modules\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.004750 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.003669    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f2206a53-85f4-42a1-bd3b-2bbd84db2978-etc-tuned\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.004750 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.003721    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-host\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.004750 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.003732    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/eccc078f-2e4c-4a45-9056-d0cd7053faf6-hosts-file\") pod \"node-resolver-fz9bt\" (UID: \"eccc078f-2e4c-4a45-9056-d0cd7053faf6\") " pod="openshift-dns/node-resolver-fz9bt"
Jul 17 09:26:45.004750 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.003725    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f357001d-cac3-4c7d-b427-7cc231e3d4ff-original-pull-secret\") pod \"global-pull-secret-syncer-trvgv\" (UID: \"f357001d-cac3-4c7d-b427-7cc231e3d4ff\") " pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:26:45.004750 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.003791    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-run\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.004750 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.003794    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-etc-sysconfig\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.004750 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.003823    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-var-lib-kubelet\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.004750 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:45.003837    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 09:26:45.004750 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.003849    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4qcj\" (UniqueName: \"kubernetes.io/projected/eccc078f-2e4c-4a45-9056-d0cd7053faf6-kube-api-access-d4qcj\") pod \"node-resolver-fz9bt\" (UID: \"eccc078f-2e4c-4a45-9056-d0cd7053faf6\") " pod="openshift-dns/node-resolver-fz9bt"
Jul 17 09:26:45.005764 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:45.003878    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f357001d-cac3-4c7d-b427-7cc231e3d4ff-original-pull-secret podName:f357001d-cac3-4c7d-b427-7cc231e3d4ff nodeName:}" failed. No retries permitted until 2026-07-17 09:26:45.503865505 +0000 UTC m=+3.086503859 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f357001d-cac3-4c7d-b427-7cc231e3d4ff-original-pull-secret") pod "global-pull-secret-syncer-trvgv" (UID: "f357001d-cac3-4c7d-b427-7cc231e3d4ff") : object "kube-system"/"original-pull-secret" not registered
Jul 17 09:26:45.005764 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.003898    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-etc-sysctl-conf\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.005764 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.003915    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eccc078f-2e4c-4a45-9056-d0cd7053faf6-tmp-dir\") pod \"node-resolver-fz9bt\" (UID: \"eccc078f-2e4c-4a45-9056-d0cd7053faf6\") " pod="openshift-dns/node-resolver-fz9bt"
Jul 17 09:26:45.005764 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.003939    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-etc-systemd\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.005764 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.003956    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f357001d-cac3-4c7d-b427-7cc231e3d4ff-kubelet-config\") pod \"global-pull-secret-syncer-trvgv\" (UID: \"f357001d-cac3-4c7d-b427-7cc231e3d4ff\") " pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:26:45.005764 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.003972    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2206a53-85f4-42a1-bd3b-2bbd84db2978-tmp\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.005764 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.003993    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-etc-kubernetes\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.005764 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.004011    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f357001d-cac3-4c7d-b427-7cc231e3d4ff-dbus\") pod \"global-pull-secret-syncer-trvgv\" (UID: \"f357001d-cac3-4c7d-b427-7cc231e3d4ff\") " pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:26:45.005764 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.004040    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-etc-sysctl-d\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.005764 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.004086    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-etc-modprobe-d\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.005764 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.004103    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-etc-sysconfig\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.005764 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.004154    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-var-lib-kubelet\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.005764 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.004199    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f357001d-cac3-4c7d-b427-7cc231e3d4ff-kubelet-config\") pod \"global-pull-secret-syncer-trvgv\" (UID: \"f357001d-cac3-4c7d-b427-7cc231e3d4ff\") " pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:26:45.005764 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.004214    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-etc-modprobe-d\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.005764 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.004031    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-sys\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.005764 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.004299    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f357001d-cac3-4c7d-b427-7cc231e3d4ff-dbus\") pod \"global-pull-secret-syncer-trvgv\" (UID: \"f357001d-cac3-4c7d-b427-7cc231e3d4ff\") " pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:26:45.005764 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.004381    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-etc-sysctl-d\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.006503 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.004406    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-etc-systemd\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.006503 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.004435    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-etc-kubernetes\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.006503 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.004512    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0f1dd372-9aca-4b16-8020-eee6e421e6a2-konnectivity-ca\") pod \"konnectivity-agent-gd59l\" (UID: \"0f1dd372-9aca-4b16-8020-eee6e421e6a2\") " pod="kube-system/konnectivity-agent-gd59l"
Jul 17 09:26:45.006503 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.004520    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f2206a53-85f4-42a1-bd3b-2bbd84db2978-etc-sysctl-conf\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.006503 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.004695    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eccc078f-2e4c-4a45-9056-d0cd7053faf6-tmp-dir\") pod \"node-resolver-fz9bt\" (UID: \"eccc078f-2e4c-4a45-9056-d0cd7053faf6\") " pod="openshift-dns/node-resolver-fz9bt"
Jul 17 09:26:45.006503 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.006357    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f2206a53-85f4-42a1-bd3b-2bbd84db2978-etc-tuned\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.006781 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.006506    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0f1dd372-9aca-4b16-8020-eee6e421e6a2-agent-certs\") pod \"konnectivity-agent-gd59l\" (UID: \"0f1dd372-9aca-4b16-8020-eee6e421e6a2\") " pod="kube-system/konnectivity-agent-gd59l"
Jul 17 09:26:45.006781 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.006610    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2206a53-85f4-42a1-bd3b-2bbd84db2978-tmp\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.012389 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.012367    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4qcj\" (UniqueName: \"kubernetes.io/projected/eccc078f-2e4c-4a45-9056-d0cd7053faf6-kube-api-access-d4qcj\") pod \"node-resolver-fz9bt\" (UID: \"eccc078f-2e4c-4a45-9056-d0cd7053faf6\") " pod="openshift-dns/node-resolver-fz9bt"
Jul 17 09:26:45.012653 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.012636    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgr7r\" (UniqueName: \"kubernetes.io/projected/f2206a53-85f4-42a1-bd3b-2bbd84db2978-kube-api-access-qgr7r\") pod \"tuned-j96bm\" (UID: \"f2206a53-85f4-42a1-bd3b-2bbd84db2978\") " pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.090721 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.090691    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4xwhf"
Jul 17 09:26:45.097554 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.097530    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4x7kz"
Jul 17 09:26:45.105009 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.104982    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-d4ksz"
Jul 17 09:26:45.109032 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.109013    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mcbxh"
Jul 17 09:26:45.110451 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:45.110387    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabec7743_4f1a_45ad_bddf_3f9c0b7d42dc.slice/crio-39453fd41904b741cc792b16b10eafe4027b9dcf8e6229341a7eda612d3d26ee WatchSource:0}: Error finding container 39453fd41904b741cc792b16b10eafe4027b9dcf8e6229341a7eda612d3d26ee: Status 404 returned error can't find the container with id 39453fd41904b741cc792b16b10eafe4027b9dcf8e6229341a7eda612d3d26ee
Jul 17 09:26:45.115207 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.114444    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:26:45.119314 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:45.119283    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c165f52_605b_48e9_9cff_bf1e0b7b97af.slice/crio-7afe8fe8b9b9bd59bdf1d2299595fa715de4f59e1866761dc84accaf78ac0542 WatchSource:0}: Error finding container 7afe8fe8b9b9bd59bdf1d2299595fa715de4f59e1866761dc84accaf78ac0542: Status 404 returned error can't find the container with id 7afe8fe8b9b9bd59bdf1d2299595fa715de4f59e1866761dc84accaf78ac0542
Jul 17 09:26:45.121144 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.120680    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25"
Jul 17 09:26:45.123956 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:45.123931    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87fc473b_6f15_4afc_8632_62937f882dbc.slice/crio-89e1eee012b2c194cd4290c38595bd87844fb39b5de87c25b24ee077b6db64a7 WatchSource:0}: Error finding container 89e1eee012b2c194cd4290c38595bd87844fb39b5de87c25b24ee077b6db64a7: Status 404 returned error can't find the container with id 89e1eee012b2c194cd4290c38595bd87844fb39b5de87c25b24ee077b6db64a7
Jul 17 09:26:45.126684 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.126645    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gd59l"
Jul 17 09:26:45.133946 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:45.133794    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19ea8f77_c130_4773_895f_d2ee2c224f10.slice/crio-9e73635dde654b3663008774c10a5febac206bd13eaa76dc06728bcfbf5d0711 WatchSource:0}: Error finding container 9e73635dde654b3663008774c10a5febac206bd13eaa76dc06728bcfbf5d0711: Status 404 returned error can't find the container with id 9e73635dde654b3663008774c10a5febac206bd13eaa76dc06728bcfbf5d0711
Jul 17 09:26:45.135481 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.135368    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j96bm"
Jul 17 09:26:45.138345 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:45.138321    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84db06ab_87e2_43b3_82d0_13f06ba86d06.slice/crio-db456f066615fe595d1e1c1043d86ff99cc009109e40331cc27bad773a9b641a WatchSource:0}: Error finding container db456f066615fe595d1e1c1043d86ff99cc009109e40331cc27bad773a9b641a: Status 404 returned error can't find the container with id db456f066615fe595d1e1c1043d86ff99cc009109e40331cc27bad773a9b641a
Jul 17 09:26:45.139940 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.139922    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fz9bt"
Jul 17 09:26:45.150412 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:45.150393    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f1dd372_9aca_4b16_8020_eee6e421e6a2.slice/crio-e10996fb68660dd06f51f2caf47efa2274b8040e791e01235730c00c1becbce5 WatchSource:0}: Error finding container e10996fb68660dd06f51f2caf47efa2274b8040e791e01235730c00c1becbce5: Status 404 returned error can't find the container with id e10996fb68660dd06f51f2caf47efa2274b8040e791e01235730c00c1becbce5
Jul 17 09:26:45.155102 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:26:45.155082    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeccc078f_2e4c_4a45_9056_d0cd7053faf6.slice/crio-8da162889409bb43144960bdc6c6a833b2ac68582c26304bbe6ad8c70fdb163d WatchSource:0}: Error finding container 8da162889409bb43144960bdc6c6a833b2ac68582c26304bbe6ad8c70fdb163d: Status 404 returned error can't find the container with id 8da162889409bb43144960bdc6c6a833b2ac68582c26304bbe6ad8c70fdb163d
Jul 17 09:26:45.407670 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.407599    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb87a089-9075-4b84-a8b6-98cfaa1d9387-metrics-certs\") pod \"network-metrics-daemon-s62sx\" (UID: \"fb87a089-9075-4b84-a8b6-98cfaa1d9387\") " pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:26:45.407806 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:45.407794    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:26:45.407870 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:45.407853    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb87a089-9075-4b84-a8b6-98cfaa1d9387-metrics-certs podName:fb87a089-9075-4b84-a8b6-98cfaa1d9387 nodeName:}" failed. No retries permitted until 2026-07-17 09:26:46.407835778 +0000 UTC m=+3.990474137 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fb87a089-9075-4b84-a8b6-98cfaa1d9387-metrics-certs") pod "network-metrics-daemon-s62sx" (UID: "fb87a089-9075-4b84-a8b6-98cfaa1d9387") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:26:45.508778 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.508750    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f357001d-cac3-4c7d-b427-7cc231e3d4ff-original-pull-secret\") pod \"global-pull-secret-syncer-trvgv\" (UID: \"f357001d-cac3-4c7d-b427-7cc231e3d4ff\") " pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:26:45.508918 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.508832    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf448\" (UniqueName: \"kubernetes.io/projected/ba08cebc-2cf5-4174-9698-031b279f9258-kube-api-access-cf448\") pod \"network-check-target-qks8f\" (UID: \"ba08cebc-2cf5-4174-9698-031b279f9258\") " pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:26:45.508991 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:45.508919    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 09:26:45.508991 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:45.508951    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 09:26:45.508991 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:45.508965    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 09:26:45.508991 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:45.508975    2574 projected.go:196] Error preparing data for projected volume kube-api-access-cf448 for pod openshift-network-diagnostics/network-check-target-qks8f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:26:45.509119 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:45.508997    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f357001d-cac3-4c7d-b427-7cc231e3d4ff-original-pull-secret podName:f357001d-cac3-4c7d-b427-7cc231e3d4ff nodeName:}" failed. No retries permitted until 2026-07-17 09:26:46.50897933 +0000 UTC m=+4.091617683 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f357001d-cac3-4c7d-b427-7cc231e3d4ff-original-pull-secret") pod "global-pull-secret-syncer-trvgv" (UID: "f357001d-cac3-4c7d-b427-7cc231e3d4ff") : object "kube-system"/"original-pull-secret" not registered
Jul 17 09:26:45.509119 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:45.509016    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba08cebc-2cf5-4174-9698-031b279f9258-kube-api-access-cf448 podName:ba08cebc-2cf5-4174-9698-031b279f9258 nodeName:}" failed. No retries permitted until 2026-07-17 09:26:46.509005055 +0000 UTC m=+4.091643409 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-cf448" (UniqueName: "kubernetes.io/projected/ba08cebc-2cf5-4174-9698-031b279f9258-kube-api-access-cf448") pod "network-check-target-qks8f" (UID: "ba08cebc-2cf5-4174-9698-031b279f9258") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:26:45.891741 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.891674    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:26:45.892398 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:45.891820    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trvgv" podUID="f357001d-cac3-4c7d-b427-7cc231e3d4ff"
Jul 17 09:26:45.901580 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.901552    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gd59l" event={"ID":"0f1dd372-9aca-4b16-8020-eee6e421e6a2","Type":"ContainerStarted","Data":"e10996fb68660dd06f51f2caf47efa2274b8040e791e01235730c00c1becbce5"}
Jul 17 09:26:45.902805 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.902776    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25" event={"ID":"19ea8f77-c130-4773-895f-d2ee2c224f10","Type":"ContainerStarted","Data":"9e73635dde654b3663008774c10a5febac206bd13eaa76dc06728bcfbf5d0711"}
Jul 17 09:26:45.904227 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.904201    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mcbxh" event={"ID":"87fc473b-6f15-4afc-8632-62937f882dbc","Type":"ContainerStarted","Data":"89e1eee012b2c194cd4290c38595bd87844fb39b5de87c25b24ee077b6db64a7"}
Jul 17 09:26:45.905786 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.905758    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fz9bt" event={"ID":"eccc078f-2e4c-4a45-9056-d0cd7053faf6","Type":"ContainerStarted","Data":"8da162889409bb43144960bdc6c6a833b2ac68582c26304bbe6ad8c70fdb163d"}
Jul 17 09:26:45.907434 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.907401    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j96bm" event={"ID":"f2206a53-85f4-42a1-bd3b-2bbd84db2978","Type":"ContainerStarted","Data":"25acfcc589d397021ca67e36bdb03baafbaaa5afb955f4ed1e7f065b7311938c"}
Jul 17 09:26:45.908884 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.908861    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dh564" event={"ID":"84db06ab-87e2-43b3-82d0-13f06ba86d06","Type":"ContainerStarted","Data":"db456f066615fe595d1e1c1043d86ff99cc009109e40331cc27bad773a9b641a"}
Jul 17 09:26:45.911172 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.911136    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d4ksz" event={"ID":"5c165f52-605b-48e9-9cff-bf1e0b7b97af","Type":"ContainerStarted","Data":"7afe8fe8b9b9bd59bdf1d2299595fa715de4f59e1866761dc84accaf78ac0542"}
Jul 17 09:26:45.912471 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.912447    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4x7kz" event={"ID":"abec7743-4f1a-45ad-bddf-3f9c0b7d42dc","Type":"ContainerStarted","Data":"39453fd41904b741cc792b16b10eafe4027b9dcf8e6229341a7eda612d3d26ee"}
Jul 17 09:26:45.915126 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.915102    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4xwhf" event={"ID":"270ef556-fd73-4a5b-bf9a-73ecab7842ca","Type":"ContainerStarted","Data":"50caa80779ec55f8f7a5340020b0cee2ccac2dbb8fc204bbc66d2ce941455f96"}
Jul 17 09:26:45.922170 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.922133    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 09:21:44 +0000 UTC" deadline="2028-04-03 01:07:51.05117943 +0000 UTC"
Jul 17 09:26:45.922170 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:45.922161    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15015h41m5.129022063s"
Jul 17 09:26:46.415204 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:46.415143    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb87a089-9075-4b84-a8b6-98cfaa1d9387-metrics-certs\") pod \"network-metrics-daemon-s62sx\" (UID: \"fb87a089-9075-4b84-a8b6-98cfaa1d9387\") " pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:26:46.415397 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:46.415296    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:26:46.415397 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:46.415369    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb87a089-9075-4b84-a8b6-98cfaa1d9387-metrics-certs podName:fb87a089-9075-4b84-a8b6-98cfaa1d9387 nodeName:}" failed. No retries permitted until 2026-07-17 09:26:48.415350858 +0000 UTC m=+5.997989222 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fb87a089-9075-4b84-a8b6-98cfaa1d9387-metrics-certs") pod "network-metrics-daemon-s62sx" (UID: "fb87a089-9075-4b84-a8b6-98cfaa1d9387") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:26:46.515584 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:46.515548    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f357001d-cac3-4c7d-b427-7cc231e3d4ff-original-pull-secret\") pod \"global-pull-secret-syncer-trvgv\" (UID: \"f357001d-cac3-4c7d-b427-7cc231e3d4ff\") " pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:26:46.515753 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:46.515646    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf448\" (UniqueName: \"kubernetes.io/projected/ba08cebc-2cf5-4174-9698-031b279f9258-kube-api-access-cf448\") pod \"network-check-target-qks8f\" (UID: \"ba08cebc-2cf5-4174-9698-031b279f9258\") " pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:26:46.515813 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:46.515801    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 09:26:46.515863 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:46.515820    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 09:26:46.515863 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:46.515847    2574 projected.go:196] Error preparing data for projected volume kube-api-access-cf448 for pod openshift-network-diagnostics/network-check-target-qks8f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:26:46.515972 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:46.515907    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba08cebc-2cf5-4174-9698-031b279f9258-kube-api-access-cf448 podName:ba08cebc-2cf5-4174-9698-031b279f9258 nodeName:}" failed. No retries permitted until 2026-07-17 09:26:48.515888625 +0000 UTC m=+6.098526993 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-cf448" (UniqueName: "kubernetes.io/projected/ba08cebc-2cf5-4174-9698-031b279f9258-kube-api-access-cf448") pod "network-check-target-qks8f" (UID: "ba08cebc-2cf5-4174-9698-031b279f9258") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:26:46.516031 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:46.515985    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 09:26:46.516031 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:46.516018    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f357001d-cac3-4c7d-b427-7cc231e3d4ff-original-pull-secret podName:f357001d-cac3-4c7d-b427-7cc231e3d4ff nodeName:}" failed. No retries permitted until 2026-07-17 09:26:48.516007078 +0000 UTC m=+6.098645446 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f357001d-cac3-4c7d-b427-7cc231e3d4ff-original-pull-secret") pod "global-pull-secret-syncer-trvgv" (UID: "f357001d-cac3-4c7d-b427-7cc231e3d4ff") : object "kube-system"/"original-pull-secret" not registered
Jul 17 09:26:46.894382 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:46.892337    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:26:46.894382 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:46.892463    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qks8f" podUID="ba08cebc-2cf5-4174-9698-031b279f9258"
Jul 17 09:26:46.894382 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:46.892921    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:26:46.894382 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:46.893032    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62sx" podUID="fb87a089-9075-4b84-a8b6-98cfaa1d9387"
Jul 17 09:26:46.922873 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:46.922795    2574 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 09:21:44 +0000 UTC" deadline="2028-04-25 08:37:21.144055998 +0000 UTC"
Jul 17 09:26:46.922873 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:46.922845    2574 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15551h10m34.221215129s"
Jul 17 09:26:47.891388 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:47.891315    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:26:47.891532 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:47.891441    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trvgv" podUID="f357001d-cac3-4c7d-b427-7cc231e3d4ff"
Jul 17 09:26:47.936425 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:47.936389    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-235.ec2.internal" event={"ID":"fa208b626e54278e56162a86ee4791d7","Type":"ContainerStarted","Data":"1f9bbca4ce0c0503ddfa43de2c8d3704e041e1414d27ff94ae3d2fa16d625d2d"}
Jul 17 09:26:47.944566 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:47.944517    2574 generic.go:356] "Generic (PLEG): container finished" podID="b5a95e7ea41ed108b0a5006ff67827fb" containerID="40c6879edd9a2756d7a148d69fb49be994a469be20686361a1bdc206e136aa59" exitCode=0
Jul 17 09:26:47.944566 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:47.944557    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-235.ec2.internal" event={"ID":"b5a95e7ea41ed108b0a5006ff67827fb","Type":"ContainerDied","Data":"40c6879edd9a2756d7a148d69fb49be994a469be20686361a1bdc206e136aa59"}
Jul 17 09:26:47.952421 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:47.952375    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-235.ec2.internal" podStartSLOduration=4.952358317 podStartE2EDuration="4.952358317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:26:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:26:47.95222888 +0000 UTC m=+5.534867259" watchObservedRunningTime="2026-07-17 09:26:47.952358317 +0000 UTC m=+5.534996714"
Jul 17 09:26:48.435302 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:48.435269    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb87a089-9075-4b84-a8b6-98cfaa1d9387-metrics-certs\") pod \"network-metrics-daemon-s62sx\" (UID: \"fb87a089-9075-4b84-a8b6-98cfaa1d9387\") " pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:26:48.435487 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:48.435426    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:26:48.435487 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:48.435486    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb87a089-9075-4b84-a8b6-98cfaa1d9387-metrics-certs podName:fb87a089-9075-4b84-a8b6-98cfaa1d9387 nodeName:}" failed. No retries permitted until 2026-07-17 09:26:52.435465133 +0000 UTC m=+10.018103490 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fb87a089-9075-4b84-a8b6-98cfaa1d9387-metrics-certs") pod "network-metrics-daemon-s62sx" (UID: "fb87a089-9075-4b84-a8b6-98cfaa1d9387") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:26:48.536220 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:48.536188    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf448\" (UniqueName: \"kubernetes.io/projected/ba08cebc-2cf5-4174-9698-031b279f9258-kube-api-access-cf448\") pod \"network-check-target-qks8f\" (UID: \"ba08cebc-2cf5-4174-9698-031b279f9258\") " pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:26:48.536393 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:48.536260    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f357001d-cac3-4c7d-b427-7cc231e3d4ff-original-pull-secret\") pod \"global-pull-secret-syncer-trvgv\" (UID: \"f357001d-cac3-4c7d-b427-7cc231e3d4ff\") " pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:26:48.536393 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:48.536378    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 09:26:48.536516 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:48.536432    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f357001d-cac3-4c7d-b427-7cc231e3d4ff-original-pull-secret podName:f357001d-cac3-4c7d-b427-7cc231e3d4ff nodeName:}" failed. No retries permitted until 2026-07-17 09:26:52.536415173 +0000 UTC m=+10.119053530 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f357001d-cac3-4c7d-b427-7cc231e3d4ff-original-pull-secret") pod "global-pull-secret-syncer-trvgv" (UID: "f357001d-cac3-4c7d-b427-7cc231e3d4ff") : object "kube-system"/"original-pull-secret" not registered
Jul 17 09:26:48.536856 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:48.536837    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 09:26:48.536944 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:48.536864    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 09:26:48.536944 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:48.536876    2574 projected.go:196] Error preparing data for projected volume kube-api-access-cf448 for pod openshift-network-diagnostics/network-check-target-qks8f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:26:48.536944 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:48.536920    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba08cebc-2cf5-4174-9698-031b279f9258-kube-api-access-cf448 podName:ba08cebc-2cf5-4174-9698-031b279f9258 nodeName:}" failed. No retries permitted until 2026-07-17 09:26:52.536905359 +0000 UTC m=+10.119543715 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-cf448" (UniqueName: "kubernetes.io/projected/ba08cebc-2cf5-4174-9698-031b279f9258-kube-api-access-cf448") pod "network-check-target-qks8f" (UID: "ba08cebc-2cf5-4174-9698-031b279f9258") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:26:48.892196 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:48.892126    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:26:48.892344 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:48.892275    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qks8f" podUID="ba08cebc-2cf5-4174-9698-031b279f9258"
Jul 17 09:26:48.892344 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:48.892309    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:26:48.892499 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:48.892454    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62sx" podUID="fb87a089-9075-4b84-a8b6-98cfaa1d9387"
Jul 17 09:26:49.890976 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:49.890942    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:26:49.891377 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:49.891065    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trvgv" podUID="f357001d-cac3-4c7d-b427-7cc231e3d4ff"
Jul 17 09:26:50.892083 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:50.891452    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:26:50.892083 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:50.891581    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qks8f" podUID="ba08cebc-2cf5-4174-9698-031b279f9258"
Jul 17 09:26:50.892083 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:50.891936    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:26:50.892083 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:50.892041    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62sx" podUID="fb87a089-9075-4b84-a8b6-98cfaa1d9387"
Jul 17 09:26:51.891387 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:51.891354    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:26:51.891545 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:51.891494    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trvgv" podUID="f357001d-cac3-4c7d-b427-7cc231e3d4ff"
Jul 17 09:26:52.465233 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:52.465160    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb87a089-9075-4b84-a8b6-98cfaa1d9387-metrics-certs\") pod \"network-metrics-daemon-s62sx\" (UID: \"fb87a089-9075-4b84-a8b6-98cfaa1d9387\") " pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:26:52.465783 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:52.465336    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:26:52.465783 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:52.465403    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb87a089-9075-4b84-a8b6-98cfaa1d9387-metrics-certs podName:fb87a089-9075-4b84-a8b6-98cfaa1d9387 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:00.465382507 +0000 UTC m=+18.048020875 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fb87a089-9075-4b84-a8b6-98cfaa1d9387-metrics-certs") pod "network-metrics-daemon-s62sx" (UID: "fb87a089-9075-4b84-a8b6-98cfaa1d9387") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:26:52.566273 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:52.565964    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f357001d-cac3-4c7d-b427-7cc231e3d4ff-original-pull-secret\") pod \"global-pull-secret-syncer-trvgv\" (UID: \"f357001d-cac3-4c7d-b427-7cc231e3d4ff\") " pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:26:52.566273 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:52.566043    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf448\" (UniqueName: \"kubernetes.io/projected/ba08cebc-2cf5-4174-9698-031b279f9258-kube-api-access-cf448\") pod \"network-check-target-qks8f\" (UID: \"ba08cebc-2cf5-4174-9698-031b279f9258\") " pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:26:52.566273 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:52.566161    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 09:26:52.566273 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:52.566174    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 09:26:52.566273 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:52.566184    2574 projected.go:196] Error preparing data for projected volume kube-api-access-cf448 for pod openshift-network-diagnostics/network-check-target-qks8f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:26:52.566273 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:52.566228    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba08cebc-2cf5-4174-9698-031b279f9258-kube-api-access-cf448 podName:ba08cebc-2cf5-4174-9698-031b279f9258 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:00.56621557 +0000 UTC m=+18.148853924 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-cf448" (UniqueName: "kubernetes.io/projected/ba08cebc-2cf5-4174-9698-031b279f9258-kube-api-access-cf448") pod "network-check-target-qks8f" (UID: "ba08cebc-2cf5-4174-9698-031b279f9258") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:26:52.566684 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:52.566639    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 09:26:52.566749 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:52.566691    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f357001d-cac3-4c7d-b427-7cc231e3d4ff-original-pull-secret podName:f357001d-cac3-4c7d-b427-7cc231e3d4ff nodeName:}" failed. No retries permitted until 2026-07-17 09:27:00.566676871 +0000 UTC m=+18.149315226 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f357001d-cac3-4c7d-b427-7cc231e3d4ff-original-pull-secret") pod "global-pull-secret-syncer-trvgv" (UID: "f357001d-cac3-4c7d-b427-7cc231e3d4ff") : object "kube-system"/"original-pull-secret" not registered
Jul 17 09:26:52.891894 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:52.891820    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:26:52.892039 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:52.891927    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qks8f" podUID="ba08cebc-2cf5-4174-9698-031b279f9258"
Jul 17 09:26:52.892426 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:52.892402    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:26:52.892552 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:52.892509    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62sx" podUID="fb87a089-9075-4b84-a8b6-98cfaa1d9387"
Jul 17 09:26:53.891422 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:53.891336    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:26:53.891859 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:53.891453    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trvgv" podUID="f357001d-cac3-4c7d-b427-7cc231e3d4ff"
Jul 17 09:26:54.891600 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:54.891067    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:26:54.891600 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:54.891187    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qks8f" podUID="ba08cebc-2cf5-4174-9698-031b279f9258"
Jul 17 09:26:54.892132 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:54.891691    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:26:54.892132 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:54.891783    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62sx" podUID="fb87a089-9075-4b84-a8b6-98cfaa1d9387"
Jul 17 09:26:55.891785 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:55.891752    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:26:55.892177 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:55.891924    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trvgv" podUID="f357001d-cac3-4c7d-b427-7cc231e3d4ff"
Jul 17 09:26:56.893369 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:56.893339    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:26:56.893785 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:56.893345    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:26:56.893785 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:56.893468    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qks8f" podUID="ba08cebc-2cf5-4174-9698-031b279f9258"
Jul 17 09:26:56.893785 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:56.893575    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62sx" podUID="fb87a089-9075-4b84-a8b6-98cfaa1d9387"
Jul 17 09:26:57.891626 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:57.891587    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:26:57.891805 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:57.891719    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trvgv" podUID="f357001d-cac3-4c7d-b427-7cc231e3d4ff"
Jul 17 09:26:58.891792 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:58.891757    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:26:58.892268 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:58.891878    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qks8f" podUID="ba08cebc-2cf5-4174-9698-031b279f9258"
Jul 17 09:26:58.892268 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:58.891917    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:26:58.892268 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:58.892022    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62sx" podUID="fb87a089-9075-4b84-a8b6-98cfaa1d9387"
Jul 17 09:26:59.891295 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:26:59.891269    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:26:59.891509 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:26:59.891377    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trvgv" podUID="f357001d-cac3-4c7d-b427-7cc231e3d4ff"
Jul 17 09:27:00.522170 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:00.522143    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb87a089-9075-4b84-a8b6-98cfaa1d9387-metrics-certs\") pod \"network-metrics-daemon-s62sx\" (UID: \"fb87a089-9075-4b84-a8b6-98cfaa1d9387\") " pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:27:00.522574 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:00.522325    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:27:00.522574 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:00.522390    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb87a089-9075-4b84-a8b6-98cfaa1d9387-metrics-certs podName:fb87a089-9075-4b84-a8b6-98cfaa1d9387 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:16.522370475 +0000 UTC m=+34.105008829 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fb87a089-9075-4b84-a8b6-98cfaa1d9387-metrics-certs") pod "network-metrics-daemon-s62sx" (UID: "fb87a089-9075-4b84-a8b6-98cfaa1d9387") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:27:00.622500 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:00.622472    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf448\" (UniqueName: \"kubernetes.io/projected/ba08cebc-2cf5-4174-9698-031b279f9258-kube-api-access-cf448\") pod \"network-check-target-qks8f\" (UID: \"ba08cebc-2cf5-4174-9698-031b279f9258\") " pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:27:00.622660 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:00.622517    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f357001d-cac3-4c7d-b427-7cc231e3d4ff-original-pull-secret\") pod \"global-pull-secret-syncer-trvgv\" (UID: \"f357001d-cac3-4c7d-b427-7cc231e3d4ff\") " pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:27:00.622660 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:00.622622    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 09:27:00.622779 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:00.622656    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 09:27:00.622779 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:00.622678    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f357001d-cac3-4c7d-b427-7cc231e3d4ff-original-pull-secret podName:f357001d-cac3-4c7d-b427-7cc231e3d4ff nodeName:}" failed. No retries permitted until 2026-07-17 09:27:16.622661126 +0000 UTC m=+34.205299480 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f357001d-cac3-4c7d-b427-7cc231e3d4ff-original-pull-secret") pod "global-pull-secret-syncer-trvgv" (UID: "f357001d-cac3-4c7d-b427-7cc231e3d4ff") : object "kube-system"/"original-pull-secret" not registered
Jul 17 09:27:00.622779 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:00.622679    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 09:27:00.622779 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:00.622698    2574 projected.go:196] Error preparing data for projected volume kube-api-access-cf448 for pod openshift-network-diagnostics/network-check-target-qks8f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:27:00.622779 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:00.622756    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba08cebc-2cf5-4174-9698-031b279f9258-kube-api-access-cf448 podName:ba08cebc-2cf5-4174-9698-031b279f9258 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:16.622739679 +0000 UTC m=+34.205378050 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-cf448" (UniqueName: "kubernetes.io/projected/ba08cebc-2cf5-4174-9698-031b279f9258-kube-api-access-cf448") pod "network-check-target-qks8f" (UID: "ba08cebc-2cf5-4174-9698-031b279f9258") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:27:00.891223 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:00.891146    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:27:00.891378 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:00.891275    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qks8f" podUID="ba08cebc-2cf5-4174-9698-031b279f9258"
Jul 17 09:27:00.891378 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:00.891323    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:27:00.891489 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:00.891399    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62sx" podUID="fb87a089-9075-4b84-a8b6-98cfaa1d9387"
Jul 17 09:27:01.891873 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:01.891847    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:27:01.892276 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:01.891961    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trvgv" podUID="f357001d-cac3-4c7d-b427-7cc231e3d4ff"
Jul 17 09:27:02.891587 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:02.891557    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:27:02.891787 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:02.891633    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:27:02.891787 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:02.891653    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qks8f" podUID="ba08cebc-2cf5-4174-9698-031b279f9258"
Jul 17 09:27:02.891787 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:02.891735    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62sx" podUID="fb87a089-9075-4b84-a8b6-98cfaa1d9387"
Jul 17 09:27:03.890980 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:03.890945    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:27:03.891431 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:03.891069    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trvgv" podUID="f357001d-cac3-4c7d-b427-7cc231e3d4ff"
Jul 17 09:27:04.890947 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:04.890922    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:27:04.891148 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:04.891050    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qks8f" podUID="ba08cebc-2cf5-4174-9698-031b279f9258"
Jul 17 09:27:04.891148 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:04.891103    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:27:04.891830 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:04.891221    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62sx" podUID="fb87a089-9075-4b84-a8b6-98cfaa1d9387"
Jul 17 09:27:04.981704 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:04.981488    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-235.ec2.internal" event={"ID":"b5a95e7ea41ed108b0a5006ff67827fb","Type":"ContainerStarted","Data":"23bfcd632d6ab10123fa92c53d8f4cd73d6fee45fae5a8b7d325ca342673ab7f"}
Jul 17 09:27:04.988302 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:04.987964    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j96bm" event={"ID":"f2206a53-85f4-42a1-bd3b-2bbd84db2978","Type":"ContainerStarted","Data":"34fffe5bf96b6aa3344683ff9c6c4a7533719e3b1c057034dc34d880e73bdbee"}
Jul 17 09:27:04.992796 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:04.992417    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gd59l" event={"ID":"0f1dd372-9aca-4b16-8020-eee6e421e6a2","Type":"ContainerStarted","Data":"376009d473cf7e345f0cf64f2c322d328cfbb0643c7987ebea99387489fb45a6"}
Jul 17 09:27:04.997956 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:04.997444    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-235.ec2.internal" podStartSLOduration=21.997425604 podStartE2EDuration="21.997425604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:26:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:04.9957421 +0000 UTC m=+22.578380480" watchObservedRunningTime="2026-07-17 09:27:04.997425604 +0000 UTC m=+22.580063980"
Jul 17 09:27:05.028894 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:05.028587    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-gd59l" podStartSLOduration=7.146721018 podStartE2EDuration="22.028573799s" totalImagesPullingTime="14.881852781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:26:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:05.010449221 +0000 UTC m=+22.593087600" watchObservedRunningTime="2026-07-17 09:27:05.028573799 +0000 UTC m=+22.611212165"
Jul 17 09:27:05.028894 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:05.028799    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-j96bm" podStartSLOduration=2.355832403 podStartE2EDuration="22.028795313s" totalImagesPullingTime="19.67296291s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:26:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:05.028557105 +0000 UTC m=+22.611195484" watchObservedRunningTime="2026-07-17 09:27:05.028795313 +0000 UTC m=+22.611433688"
Jul 17 09:27:05.891991 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:05.891812    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:27:05.892633 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:05.892071    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trvgv" podUID="f357001d-cac3-4c7d-b427-7cc231e3d4ff"
Jul 17 09:27:05.996214 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:05.996135    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mcbxh" event={"ID":"87fc473b-6f15-4afc-8632-62937f882dbc","Type":"ContainerStarted","Data":"c1098cef6631fb75a33c450dbf295819a7cdf34e82806b22d55795a91d0d7bd9"}
Jul 17 09:27:05.997539 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:05.997512    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fz9bt" event={"ID":"eccc078f-2e4c-4a45-9056-d0cd7053faf6","Type":"ContainerStarted","Data":"c688e32f84e019db4c088dd8d11a4ef7adb3e722581dbff211d1a4525be13fef"}
Jul 17 09:27:05.999988 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:05.999971    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dh564_84db06ab-87e2-43b3-82d0-13f06ba86d06/ovn-acl-logging/0.log"
Jul 17 09:27:06.000325 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:06.000304    2574 generic.go:356] "Generic (PLEG): container finished" podID="84db06ab-87e2-43b3-82d0-13f06ba86d06" containerID="27cd131441b65719e2b1dc0f7f0bc90512c9ce808160b868a1e75563656f975a" exitCode=1
Jul 17 09:27:06.000400 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:06.000374    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dh564" event={"ID":"84db06ab-87e2-43b3-82d0-13f06ba86d06","Type":"ContainerStarted","Data":"014cdf049f35e4bff18083f734db80b6e6134b5b6af7fe74b350aace66d76760"}
Jul 17 09:27:06.000472 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:06.000458    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dh564" event={"ID":"84db06ab-87e2-43b3-82d0-13f06ba86d06","Type":"ContainerStarted","Data":"0705799b30da43741567b2dfab6cbf05e3adfce5bb931b571da4b71a338fdacd"}
Jul 17 09:27:06.000522 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:06.000503    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dh564" event={"ID":"84db06ab-87e2-43b3-82d0-13f06ba86d06","Type":"ContainerStarted","Data":"c575e8f10e13721b765c3c53e40afddf8ccd3ec6a64913717476e2ef7ab4759b"}
Jul 17 09:27:06.000570 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:06.000543    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dh564" event={"ID":"84db06ab-87e2-43b3-82d0-13f06ba86d06","Type":"ContainerDied","Data":"27cd131441b65719e2b1dc0f7f0bc90512c9ce808160b868a1e75563656f975a"}
Jul 17 09:27:06.000621 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:06.000584    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dh564" event={"ID":"84db06ab-87e2-43b3-82d0-13f06ba86d06","Type":"ContainerStarted","Data":"d6a278bcabeb67fb7e07ebf843f218a476093fe34364fbecd4d62e2b00f5a0f6"}
Jul 17 09:27:06.001966 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:06.001938    2574 generic.go:356] "Generic (PLEG): container finished" podID="5c165f52-605b-48e9-9cff-bf1e0b7b97af" containerID="b4d49e884bda56716cd90f3929ec9c175250eda128edfe5909ec29784dd15eba" exitCode=0
Jul 17 09:27:06.002065 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:06.002011    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d4ksz" event={"ID":"5c165f52-605b-48e9-9cff-bf1e0b7b97af","Type":"ContainerDied","Data":"b4d49e884bda56716cd90f3929ec9c175250eda128edfe5909ec29784dd15eba"}
Jul 17 09:27:06.005760 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:06.005729    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4x7kz" event={"ID":"abec7743-4f1a-45ad-bddf-3f9c0b7d42dc","Type":"ContainerStarted","Data":"540d17aca3eb6d2a5b11f4317a894d170ca29d3afdc2f0257a90f5c9e7f57226"}
Jul 17 09:27:06.007642 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:06.007613    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25" event={"ID":"19ea8f77-c130-4773-895f-d2ee2c224f10","Type":"ContainerStarted","Data":"4da7e9a5dbf6549a41a5d5cdb3299d4ddcdc8a12ffa623eba5fc22132f8eff75"}
Jul 17 09:27:06.012622 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:06.012587    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mcbxh" podStartSLOduration=3.295562177 podStartE2EDuration="23.01257645s" totalImagesPullingTime="19.717014273s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:26:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:06.01237838 +0000 UTC m=+23.595016756" watchObservedRunningTime="2026-07-17 09:27:06.01257645 +0000 UTC m=+23.595214827"
Jul 17 09:27:06.025899 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:06.025867    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fz9bt" podStartSLOduration=3.364058199 podStartE2EDuration="23.025857431s" totalImagesPullingTime="19.661799232s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:26:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:06.025765195 +0000 UTC m=+23.608403571" watchObservedRunningTime="2026-07-17 09:27:06.025857431 +0000 UTC m=+23.608495810"
Jul 17 09:27:06.059487 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:06.059459    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4x7kz" podStartSLOduration=4.352027808 podStartE2EDuration="24.059448056s" totalImagesPullingTime="19.707420248s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:26:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:06.039569423 +0000 UTC m=+23.622207799" watchObservedRunningTime="2026-07-17 09:27:06.059448056 +0000 UTC m=+23.642086424"
Jul 17 09:27:06.431580 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:06.431555    2574 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 17 09:27:06.869351 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:06.869206    2574 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-17T09:27:06.431576601Z","UUID":"a732ae67-b2d3-4fa3-93cb-faf6a4ee3892","Handler":null,"Name":"","Endpoint":""}
Jul 17 09:27:06.870728 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:06.870709    2574 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 17 09:27:06.870854 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:06.870748    2574 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 17 09:27:06.891071 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:06.891049    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:27:06.891071 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:06.891060    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:27:06.891260 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:06.891165    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qks8f" podUID="ba08cebc-2cf5-4174-9698-031b279f9258"
Jul 17 09:27:06.891329 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:06.891280    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62sx" podUID="fb87a089-9075-4b84-a8b6-98cfaa1d9387"
Jul 17 09:27:07.012730 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:07.012703    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dh564_84db06ab-87e2-43b3-82d0-13f06ba86d06/ovn-acl-logging/0.log"
Jul 17 09:27:07.013223 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:07.013091    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dh564" event={"ID":"84db06ab-87e2-43b3-82d0-13f06ba86d06","Type":"ContainerStarted","Data":"b06bf82b52debaf02a38cdda9c364ce3b5abadcf76513da56306c7b3aabbb545"}
Jul 17 09:27:07.014377 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:07.014354    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4xwhf" event={"ID":"270ef556-fd73-4a5b-bf9a-73ecab7842ca","Type":"ContainerStarted","Data":"ffe010eb90a63da05046693be21728810235d56069912816b1ad18577fd4be2b"}
Jul 17 09:27:07.016094 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:07.016075    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25" event={"ID":"19ea8f77-c130-4773-895f-d2ee2c224f10","Type":"ContainerStarted","Data":"83f8cd6e293b9fedc7e99523be94bb5ca636833ad3b2dd64b60c8992ec3ec193"}
Jul 17 09:27:07.028147 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:07.028107    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4xwhf" podStartSLOduration=5.31267506 podStartE2EDuration="25.028096562s" totalImagesPullingTime="19.715421502s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:26:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:07.027960128 +0000 UTC m=+24.610598504" watchObservedRunningTime="2026-07-17 09:27:07.028096562 +0000 UTC m=+24.610734938"
Jul 17 09:27:07.891767 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:07.891736    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:27:07.892041 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:07.891864    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trvgv" podUID="f357001d-cac3-4c7d-b427-7cc231e3d4ff"
Jul 17 09:27:08.891945 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:08.891760    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:27:08.892408 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:08.892045    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62sx" podUID="fb87a089-9075-4b84-a8b6-98cfaa1d9387"
Jul 17 09:27:08.892408 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:08.891786    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:27:08.892408 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:08.892124    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qks8f" podUID="ba08cebc-2cf5-4174-9698-031b279f9258"
Jul 17 09:27:08.918197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:08.918177    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-gd59l"
Jul 17 09:27:08.918676 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:08.918658    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-gd59l"
Jul 17 09:27:09.023166 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:09.023141    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dh564_84db06ab-87e2-43b3-82d0-13f06ba86d06/ovn-acl-logging/0.log"
Jul 17 09:27:09.023573 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:09.023553    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dh564" event={"ID":"84db06ab-87e2-43b3-82d0-13f06ba86d06","Type":"ContainerStarted","Data":"b5830e050d2ba0d13995bbf8a992b0c5c05189dfb88591a0748205fb4a06aea8"}
Jul 17 09:27:09.025655 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:09.025638    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25" event={"ID":"19ea8f77-c130-4773-895f-d2ee2c224f10","Type":"ContainerStarted","Data":"e1c10c08f65c8a287da767fa40c09e69606bb7fde035678883f2630dac931f85"}
Jul 17 09:27:09.025836 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:09.025824    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-gd59l"
Jul 17 09:27:09.026320 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:09.026304    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-gd59l"
Jul 17 09:27:09.058472 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:09.058432    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-svq25" podStartSLOduration=3.519532772 podStartE2EDuration="26.058419856s" totalImagesPullingTime="22.538887084s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:26:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:09.045589987 +0000 UTC m=+26.628228363" watchObservedRunningTime="2026-07-17 09:27:09.058419856 +0000 UTC m=+26.641058246"
Jul 17 09:27:09.890868 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:09.890839    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:27:09.891075 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:09.890955    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trvgv" podUID="f357001d-cac3-4c7d-b427-7cc231e3d4ff"
Jul 17 09:27:10.891267 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:10.891169    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:27:10.891708 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:10.891303    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:27:10.891708 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:10.891317    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qks8f" podUID="ba08cebc-2cf5-4174-9698-031b279f9258"
Jul 17 09:27:10.891708 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:10.891452    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62sx" podUID="fb87a089-9075-4b84-a8b6-98cfaa1d9387"
Jul 17 09:27:11.890977 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:11.890822    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:27:11.891067 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:11.891048    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trvgv" podUID="f357001d-cac3-4c7d-b427-7cc231e3d4ff"
Jul 17 09:27:12.035312 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:12.035293    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dh564_84db06ab-87e2-43b3-82d0-13f06ba86d06/ovn-acl-logging/0.log"
Jul 17 09:27:12.035851 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:12.035641    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dh564" event={"ID":"84db06ab-87e2-43b3-82d0-13f06ba86d06","Type":"ContainerStarted","Data":"87ea9833320eb74da28783fce1765e55ddcd98e19ba20900bf3a6b34e60f42e9"}
Jul 17 09:27:12.036073 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:12.036053    2574 scope.go:117] "RemoveContainer" containerID="27cd131441b65719e2b1dc0f7f0bc90512c9ce808160b868a1e75563656f975a"
Jul 17 09:27:12.036169 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:12.036058    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:27:12.036224 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:12.036216    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:27:12.037671 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:12.037649    2574 generic.go:356] "Generic (PLEG): container finished" podID="5c165f52-605b-48e9-9cff-bf1e0b7b97af" containerID="064c43fb9402aeb03f5fb3b363d83e05c68ca3f8c39ed8adee26c5dbe16e6178" exitCode=0
Jul 17 09:27:12.037772 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:12.037694    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d4ksz" event={"ID":"5c165f52-605b-48e9-9cff-bf1e0b7b97af","Type":"ContainerDied","Data":"064c43fb9402aeb03f5fb3b363d83e05c68ca3f8c39ed8adee26c5dbe16e6178"}
Jul 17 09:27:12.051070 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:12.051053    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:27:12.892186 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:12.892155    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:27:12.892469 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:12.892382    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:27:12.892539 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:12.892462    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qks8f" podUID="ba08cebc-2cf5-4174-9698-031b279f9258"
Jul 17 09:27:12.892610 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:12.892427    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62sx" podUID="fb87a089-9075-4b84-a8b6-98cfaa1d9387"
Jul 17 09:27:13.043665 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:13.043636    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dh564_84db06ab-87e2-43b3-82d0-13f06ba86d06/ovn-acl-logging/0.log"
Jul 17 09:27:13.044181 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:13.043996    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dh564" event={"ID":"84db06ab-87e2-43b3-82d0-13f06ba86d06","Type":"ContainerStarted","Data":"2b1f673bb2b6d7106bd84dd10743b3060bb6ac42fa215578a5399b590306db9e"}
Jul 17 09:27:13.044258 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:13.044233    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:27:13.059154 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:13.059128    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:27:13.071083 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:13.071042    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dh564" podStartSLOduration=9.938252079 podStartE2EDuration="30.071029115s" totalImagesPullingTime="20.132777036s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:26:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:13.070618043 +0000 UTC m=+30.653256419" watchObservedRunningTime="2026-07-17 09:27:13.071029115 +0000 UTC m=+30.653667490"
Jul 17 09:27:13.391393 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:13.391355    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-trvgv"]
Jul 17 09:27:13.391501 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:13.391494    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:27:13.391589 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:13.391576    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trvgv" podUID="f357001d-cac3-4c7d-b427-7cc231e3d4ff"
Jul 17 09:27:13.394843 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:13.394819    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qks8f"]
Jul 17 09:27:13.395107 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:13.394910    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:27:13.395107 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:13.394985    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qks8f" podUID="ba08cebc-2cf5-4174-9698-031b279f9258"
Jul 17 09:27:13.395527 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:13.395502    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s62sx"]
Jul 17 09:27:13.395644 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:13.395620    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:27:13.395753 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:13.395733    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62sx" podUID="fb87a089-9075-4b84-a8b6-98cfaa1d9387"
Jul 17 09:27:14.048097 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:14.048066    2574 generic.go:356] "Generic (PLEG): container finished" podID="5c165f52-605b-48e9-9cff-bf1e0b7b97af" containerID="cbda3895ce73d84cd8747aeba401c301d398b337f940725caf654bf7adee5af0" exitCode=0
Jul 17 09:27:14.048474 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:14.048151    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d4ksz" event={"ID":"5c165f52-605b-48e9-9cff-bf1e0b7b97af","Type":"ContainerDied","Data":"cbda3895ce73d84cd8747aeba401c301d398b337f940725caf654bf7adee5af0"}
Jul 17 09:27:14.891344 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:14.891314    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:27:14.891491 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:14.891320    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:27:14.891491 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:14.891428    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trvgv" podUID="f357001d-cac3-4c7d-b427-7cc231e3d4ff"
Jul 17 09:27:14.891566 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:14.891327    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:27:14.891566 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:14.891509    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62sx" podUID="fb87a089-9075-4b84-a8b6-98cfaa1d9387"
Jul 17 09:27:14.891633 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:14.891577    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qks8f" podUID="ba08cebc-2cf5-4174-9698-031b279f9258"
Jul 17 09:27:16.055101 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:16.054918    2574 generic.go:356] "Generic (PLEG): container finished" podID="5c165f52-605b-48e9-9cff-bf1e0b7b97af" containerID="4929c94d8e4523fc42ba4c83603297ab56f7859eb02b94ec425b79a9f7e5c6a4" exitCode=0
Jul 17 09:27:16.055589 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:16.054996    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d4ksz" event={"ID":"5c165f52-605b-48e9-9cff-bf1e0b7b97af","Type":"ContainerDied","Data":"4929c94d8e4523fc42ba4c83603297ab56f7859eb02b94ec425b79a9f7e5c6a4"}
Jul 17 09:27:16.541009 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:16.540987    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb87a089-9075-4b84-a8b6-98cfaa1d9387-metrics-certs\") pod \"network-metrics-daemon-s62sx\" (UID: \"fb87a089-9075-4b84-a8b6-98cfaa1d9387\") " pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:27:16.541109 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:16.541096    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:27:16.541160 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:16.541141    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb87a089-9075-4b84-a8b6-98cfaa1d9387-metrics-certs podName:fb87a089-9075-4b84-a8b6-98cfaa1d9387 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:48.541128714 +0000 UTC m=+66.123767072 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fb87a089-9075-4b84-a8b6-98cfaa1d9387-metrics-certs") pod "network-metrics-daemon-s62sx" (UID: "fb87a089-9075-4b84-a8b6-98cfaa1d9387") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 17 09:27:16.641517 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:16.641494    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f357001d-cac3-4c7d-b427-7cc231e3d4ff-original-pull-secret\") pod \"global-pull-secret-syncer-trvgv\" (UID: \"f357001d-cac3-4c7d-b427-7cc231e3d4ff\") " pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:27:16.641608 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:16.641542    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf448\" (UniqueName: \"kubernetes.io/projected/ba08cebc-2cf5-4174-9698-031b279f9258-kube-api-access-cf448\") pod \"network-check-target-qks8f\" (UID: \"ba08cebc-2cf5-4174-9698-031b279f9258\") " pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:27:16.641647 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:16.641618    2574 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 17 09:27:16.641647 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:16.641623    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 17 09:27:16.641647 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:16.641640    2574 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 17 09:27:16.641647 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:16.641649    2574 projected.go:196] Error preparing data for projected volume kube-api-access-cf448 for pod openshift-network-diagnostics/network-check-target-qks8f: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:27:16.641774 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:16.641665    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f357001d-cac3-4c7d-b427-7cc231e3d4ff-original-pull-secret podName:f357001d-cac3-4c7d-b427-7cc231e3d4ff nodeName:}" failed. No retries permitted until 2026-07-17 09:27:48.641650951 +0000 UTC m=+66.224289304 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f357001d-cac3-4c7d-b427-7cc231e3d4ff-original-pull-secret") pod "global-pull-secret-syncer-trvgv" (UID: "f357001d-cac3-4c7d-b427-7cc231e3d4ff") : object "kube-system"/"original-pull-secret" not registered
Jul 17 09:27:16.641774 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:16.641681    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba08cebc-2cf5-4174-9698-031b279f9258-kube-api-access-cf448 podName:ba08cebc-2cf5-4174-9698-031b279f9258 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:48.641670337 +0000 UTC m=+66.224308691 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-cf448" (UniqueName: "kubernetes.io/projected/ba08cebc-2cf5-4174-9698-031b279f9258-kube-api-access-cf448") pod "network-check-target-qks8f" (UID: "ba08cebc-2cf5-4174-9698-031b279f9258") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 17 09:27:16.891049 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:16.890978    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:27:16.891049 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:16.891037    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:27:16.891206 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:16.891042    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:27:16.891206 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:16.891122    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-trvgv" podUID="f357001d-cac3-4c7d-b427-7cc231e3d4ff"
Jul 17 09:27:16.891313 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:16.891291    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-s62sx" podUID="fb87a089-9075-4b84-a8b6-98cfaa1d9387"
Jul 17 09:27:16.891399 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:16.891380    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qks8f" podUID="ba08cebc-2cf5-4174-9698-031b279f9258"
Jul 17 09:27:18.774792 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.774698    2574 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-235.ec2.internal" event="NodeReady"
Jul 17 09:27:18.775332 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.774839    2574 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 17 09:27:18.810647 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.810619    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-75c884f5f7-7k78v"]
Jul 17 09:27:18.812733 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.812705    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:18.815508 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.815489    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 17 09:27:18.815619 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.815532    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 17 09:27:18.815689 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.815677    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-45kr4\""
Jul 17 09:27:18.815808 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.815791    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 17 09:27:18.821110 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.821092    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 17 09:27:18.822559 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.822540    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-vqvdn"]
Jul 17 09:27:18.824524 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.824496    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vqvdn"
Jul 17 09:27:18.825381 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.825359    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75c884f5f7-7k78v"]
Jul 17 09:27:18.827412 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.827281    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 17 09:27:18.827412 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.827294    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 17 09:27:18.827412 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.827309    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5qcn8\""
Jul 17 09:27:18.827412 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.827294    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 17 09:27:18.829532 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.829507    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gdlqf"]
Jul 17 09:27:18.832864 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.832847    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gdlqf"
Jul 17 09:27:18.835049 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.834760    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vqvdn"]
Jul 17 09:27:18.835708 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.835691    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 17 09:27:18.835958 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.835941    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-p8vfs\""
Jul 17 09:27:18.836017 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.835999    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 17 09:27:18.844692 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.844663    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gdlqf"]
Jul 17 09:27:18.891046 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.891023    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:27:18.891046 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.891044    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:27:18.891255 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.891033    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:27:18.893642 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.893615    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 17 09:27:18.893917 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.893899    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qg8vm\""
Jul 17 09:27:18.894023 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.893933    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 17 09:27:18.894117 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.894097    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 17 09:27:18.894182 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.894142    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 17 09:27:18.894263 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.894227    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jxtjz\""
Jul 17 09:27:18.957916 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.957889    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gd59l_0f1dd372-9aca-4b16-8020-eee6e421e6a2/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:18.959784 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.959765    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/27c6d907-d449-4202-8700-dc369dadb9f4-registry-certificates\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:18.959886 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.959798    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/27c6d907-d449-4202-8700-dc369dadb9f4-installation-pull-secrets\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:18.959886 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.959826    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78c11134-f4c9-4e8a-b87c-3ea12f21e6d1-cert\") pod \"ingress-canary-vqvdn\" (UID: \"78c11134-f4c9-4e8a-b87c-3ea12f21e6d1\") " pod="openshift-ingress-canary/ingress-canary-vqvdn"
Jul 17 09:27:18.959979 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.959885    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/27c6d907-d449-4202-8700-dc369dadb9f4-ca-trust-extracted\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:18.959979 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.959919    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bslf8\" (UniqueName: \"kubernetes.io/projected/fdac6d65-9077-4012-bcfb-b19556f35d5f-kube-api-access-bslf8\") pod \"dns-default-gdlqf\" (UID: \"fdac6d65-9077-4012-bcfb-b19556f35d5f\") " pod="openshift-dns/dns-default-gdlqf"
Jul 17 09:27:18.959979 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.959966    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/27c6d907-d449-4202-8700-dc369dadb9f4-trusted-ca\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:18.960087 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.960009    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdac6d65-9077-4012-bcfb-b19556f35d5f-tmp-dir\") pod \"dns-default-gdlqf\" (UID: \"fdac6d65-9077-4012-bcfb-b19556f35d5f\") " pod="openshift-dns/dns-default-gdlqf"
Jul 17 09:27:18.960087 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.960068    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/27c6d907-d449-4202-8700-dc369dadb9f4-image-registry-private-configuration\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:18.960161 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.960111    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-registry-tls\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:18.960161 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.960133    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-bound-sa-token\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:18.960308 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.960157    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fdac6d65-9077-4012-bcfb-b19556f35d5f-config-volume\") pod \"dns-default-gdlqf\" (UID: \"fdac6d65-9077-4012-bcfb-b19556f35d5f\") " pod="openshift-dns/dns-default-gdlqf"
Jul 17 09:27:18.960308 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.960213    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l85jk\" (UniqueName: \"kubernetes.io/projected/78c11134-f4c9-4e8a-b87c-3ea12f21e6d1-kube-api-access-l85jk\") pod \"ingress-canary-vqvdn\" (UID: \"78c11134-f4c9-4e8a-b87c-3ea12f21e6d1\") " pod="openshift-ingress-canary/ingress-canary-vqvdn"
Jul 17 09:27:18.960308 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.960281    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj6hb\" (UniqueName: \"kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-kube-api-access-kj6hb\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:18.960308 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:18.960299    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdac6d65-9077-4012-bcfb-b19556f35d5f-metrics-tls\") pod \"dns-default-gdlqf\" (UID: \"fdac6d65-9077-4012-bcfb-b19556f35d5f\") " pod="openshift-dns/dns-default-gdlqf"
Jul 17 09:27:19.041092 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.041033    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gd59l_0f1dd372-9aca-4b16-8020-eee6e421e6a2/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:19.061013 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.060987    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/27c6d907-d449-4202-8700-dc369dadb9f4-ca-trust-extracted\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:19.061122 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.061021    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bslf8\" (UniqueName: \"kubernetes.io/projected/fdac6d65-9077-4012-bcfb-b19556f35d5f-kube-api-access-bslf8\") pod \"dns-default-gdlqf\" (UID: \"fdac6d65-9077-4012-bcfb-b19556f35d5f\") " pod="openshift-dns/dns-default-gdlqf"
Jul 17 09:27:19.061122 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.061041    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/27c6d907-d449-4202-8700-dc369dadb9f4-trusted-ca\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:19.061122 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.061062    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdac6d65-9077-4012-bcfb-b19556f35d5f-tmp-dir\") pod \"dns-default-gdlqf\" (UID: \"fdac6d65-9077-4012-bcfb-b19556f35d5f\") " pod="openshift-dns/dns-default-gdlqf"
Jul 17 09:27:19.061122 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.061108    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/27c6d907-d449-4202-8700-dc369dadb9f4-image-registry-private-configuration\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:19.061363 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.061164    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-registry-tls\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:19.061363 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.061189    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-bound-sa-token\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:19.061363 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.061214    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fdac6d65-9077-4012-bcfb-b19556f35d5f-config-volume\") pod \"dns-default-gdlqf\" (UID: \"fdac6d65-9077-4012-bcfb-b19556f35d5f\") " pod="openshift-dns/dns-default-gdlqf"
Jul 17 09:27:19.061363 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.061277    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l85jk\" (UniqueName: \"kubernetes.io/projected/78c11134-f4c9-4e8a-b87c-3ea12f21e6d1-kube-api-access-l85jk\") pod \"ingress-canary-vqvdn\" (UID: \"78c11134-f4c9-4e8a-b87c-3ea12f21e6d1\") " pod="openshift-ingress-canary/ingress-canary-vqvdn"
Jul 17 09:27:19.061363 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.061329    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kj6hb\" (UniqueName: \"kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-kube-api-access-kj6hb\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:19.061600 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.061431    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/27c6d907-d449-4202-8700-dc369dadb9f4-ca-trust-extracted\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:19.061649 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.061624    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdac6d65-9077-4012-bcfb-b19556f35d5f-metrics-tls\") pod \"dns-default-gdlqf\" (UID: \"fdac6d65-9077-4012-bcfb-b19556f35d5f\") " pod="openshift-dns/dns-default-gdlqf"
Jul 17 09:27:19.061733 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.061702    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/27c6d907-d449-4202-8700-dc369dadb9f4-registry-certificates\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:19.061823 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.061753    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/27c6d907-d449-4202-8700-dc369dadb9f4-installation-pull-secrets\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:19.061823 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.061779    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78c11134-f4c9-4e8a-b87c-3ea12f21e6d1-cert\") pod \"ingress-canary-vqvdn\" (UID: \"78c11134-f4c9-4e8a-b87c-3ea12f21e6d1\") " pod="openshift-ingress-canary/ingress-canary-vqvdn"
Jul 17 09:27:19.062020 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.062000    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdac6d65-9077-4012-bcfb-b19556f35d5f-tmp-dir\") pod \"dns-default-gdlqf\" (UID: \"fdac6d65-9077-4012-bcfb-b19556f35d5f\") " pod="openshift-dns/dns-default-gdlqf"
Jul 17 09:27:19.062102 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:19.062086    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 09:27:19.062159 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:19.062142    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdac6d65-9077-4012-bcfb-b19556f35d5f-metrics-tls podName:fdac6d65-9077-4012-bcfb-b19556f35d5f nodeName:}" failed. No retries permitted until 2026-07-17 09:27:19.562124636 +0000 UTC m=+37.144762994 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fdac6d65-9077-4012-bcfb-b19556f35d5f-metrics-tls") pod "dns-default-gdlqf" (UID: "fdac6d65-9077-4012-bcfb-b19556f35d5f") : secret "dns-default-metrics-tls" not found
Jul 17 09:27:19.062282 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:19.062267    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 09:27:19.062352 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:19.062285    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75c884f5f7-7k78v: secret "image-registry-tls" not found
Jul 17 09:27:19.062352 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:19.062344    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-registry-tls podName:27c6d907-d449-4202-8700-dc369dadb9f4 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:19.562325457 +0000 UTC m=+37.144963813 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-registry-tls") pod "image-registry-75c884f5f7-7k78v" (UID: "27c6d907-d449-4202-8700-dc369dadb9f4") : secret "image-registry-tls" not found
Jul 17 09:27:19.062456 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.062279    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/27c6d907-d449-4202-8700-dc369dadb9f4-registry-certificates\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:19.062593 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.062573    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/27c6d907-d449-4202-8700-dc369dadb9f4-trusted-ca\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:19.063509 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.062966    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fdac6d65-9077-4012-bcfb-b19556f35d5f-config-volume\") pod \"dns-default-gdlqf\" (UID: \"fdac6d65-9077-4012-bcfb-b19556f35d5f\") " pod="openshift-dns/dns-default-gdlqf"
Jul 17 09:27:19.066517 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.066493    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/27c6d907-d449-4202-8700-dc369dadb9f4-image-registry-private-configuration\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:19.066590 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.066530    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78c11134-f4c9-4e8a-b87c-3ea12f21e6d1-cert\") pod \"ingress-canary-vqvdn\" (UID: \"78c11134-f4c9-4e8a-b87c-3ea12f21e6d1\") " pod="openshift-ingress-canary/ingress-canary-vqvdn"
Jul 17 09:27:19.066638 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.066601    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/27c6d907-d449-4202-8700-dc369dadb9f4-installation-pull-secrets\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:19.069372 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.069332    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l85jk\" (UniqueName: \"kubernetes.io/projected/78c11134-f4c9-4e8a-b87c-3ea12f21e6d1-kube-api-access-l85jk\") pod \"ingress-canary-vqvdn\" (UID: \"78c11134-f4c9-4e8a-b87c-3ea12f21e6d1\") " pod="openshift-ingress-canary/ingress-canary-vqvdn"
Jul 17 09:27:19.069372 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.069350    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-bound-sa-token\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:19.069527 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.069497    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bslf8\" (UniqueName: \"kubernetes.io/projected/fdac6d65-9077-4012-bcfb-b19556f35d5f-kube-api-access-bslf8\") pod \"dns-default-gdlqf\" (UID: \"fdac6d65-9077-4012-bcfb-b19556f35d5f\") " pod="openshift-dns/dns-default-gdlqf"
Jul 17 09:27:19.074571 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.074553    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj6hb\" (UniqueName: \"kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-kube-api-access-kj6hb\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:19.136069 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.136050    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vqvdn"
Jul 17 09:27:19.280086 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.279872    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vqvdn"]
Jul 17 09:27:19.286421 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:27:19.286394    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78c11134_f4c9_4e8a_b87c_3ea12f21e6d1.slice/crio-d4312a29afd7c2283abf7927cc81bc39fa24ee13e7d65544a68ddd0759b02127 WatchSource:0}: Error finding container d4312a29afd7c2283abf7927cc81bc39fa24ee13e7d65544a68ddd0759b02127: Status 404 returned error can't find the container with id d4312a29afd7c2283abf7927cc81bc39fa24ee13e7d65544a68ddd0759b02127
Jul 17 09:27:19.565985 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.565956    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdac6d65-9077-4012-bcfb-b19556f35d5f-metrics-tls\") pod \"dns-default-gdlqf\" (UID: \"fdac6d65-9077-4012-bcfb-b19556f35d5f\") " pod="openshift-dns/dns-default-gdlqf"
Jul 17 09:27:19.566139 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.566025    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-registry-tls\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:19.566139 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:19.566118    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 09:27:19.566139 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:19.566136    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 09:27:19.566325 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:19.566148    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75c884f5f7-7k78v: secret "image-registry-tls" not found
Jul 17 09:27:19.566325 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:19.566188    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdac6d65-9077-4012-bcfb-b19556f35d5f-metrics-tls podName:fdac6d65-9077-4012-bcfb-b19556f35d5f nodeName:}" failed. No retries permitted until 2026-07-17 09:27:20.56616803 +0000 UTC m=+38.148806400 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fdac6d65-9077-4012-bcfb-b19556f35d5f-metrics-tls") pod "dns-default-gdlqf" (UID: "fdac6d65-9077-4012-bcfb-b19556f35d5f") : secret "dns-default-metrics-tls" not found
Jul 17 09:27:19.566325 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:19.566208    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-registry-tls podName:27c6d907-d449-4202-8700-dc369dadb9f4 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:20.566197053 +0000 UTC m=+38.148835406 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-registry-tls") pod "image-registry-75c884f5f7-7k78v" (UID: "27c6d907-d449-4202-8700-dc369dadb9f4") : secret "image-registry-tls" not found
Jul 17 09:27:19.845348 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.845261    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gd59l_0f1dd372-9aca-4b16-8020-eee6e421e6a2/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:19.948344 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:19.948303    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gd59l_0f1dd372-9aca-4b16-8020-eee6e421e6a2/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:20.066635 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:20.066600    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vqvdn" event={"ID":"78c11134-f4c9-4e8a-b87c-3ea12f21e6d1","Type":"ContainerStarted","Data":"d4312a29afd7c2283abf7927cc81bc39fa24ee13e7d65544a68ddd0759b02127"}
Jul 17 09:27:20.330788 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:20.330749    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gd59l_0f1dd372-9aca-4b16-8020-eee6e421e6a2/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:20.430483 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:20.430450    2574 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gd59l_0f1dd372-9aca-4b16-8020-eee6e421e6a2/konnectivity-agent/0.log" limit=1024
Jul 17 09:27:20.574687 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:20.574653    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-registry-tls\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:20.574836 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:20.574738    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdac6d65-9077-4012-bcfb-b19556f35d5f-metrics-tls\") pod \"dns-default-gdlqf\" (UID: \"fdac6d65-9077-4012-bcfb-b19556f35d5f\") " pod="openshift-dns/dns-default-gdlqf"
Jul 17 09:27:20.574836 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:20.574801    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 09:27:20.574836 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:20.574817    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75c884f5f7-7k78v: secret "image-registry-tls" not found
Jul 17 09:27:20.574957 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:20.574838    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 09:27:20.574957 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:20.574873    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-registry-tls podName:27c6d907-d449-4202-8700-dc369dadb9f4 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:22.574858444 +0000 UTC m=+40.157496802 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-registry-tls") pod "image-registry-75c884f5f7-7k78v" (UID: "27c6d907-d449-4202-8700-dc369dadb9f4") : secret "image-registry-tls" not found
Jul 17 09:27:20.574957 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:20.574888    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdac6d65-9077-4012-bcfb-b19556f35d5f-metrics-tls podName:fdac6d65-9077-4012-bcfb-b19556f35d5f nodeName:}" failed. No retries permitted until 2026-07-17 09:27:22.574881655 +0000 UTC m=+40.157520009 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fdac6d65-9077-4012-bcfb-b19556f35d5f-metrics-tls") pod "dns-default-gdlqf" (UID: "fdac6d65-9077-4012-bcfb-b19556f35d5f") : secret "dns-default-metrics-tls" not found
Jul 17 09:27:22.075056 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:22.075022    2574 generic.go:356] "Generic (PLEG): container finished" podID="5c165f52-605b-48e9-9cff-bf1e0b7b97af" containerID="6a90a6a08250cae86bb5799995ffe590065d7d3ae9a6bf824ee0e0e7e934eecd" exitCode=0
Jul 17 09:27:22.075479 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:22.075081    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d4ksz" event={"ID":"5c165f52-605b-48e9-9cff-bf1e0b7b97af","Type":"ContainerDied","Data":"6a90a6a08250cae86bb5799995ffe590065d7d3ae9a6bf824ee0e0e7e934eecd"}
Jul 17 09:27:22.591681 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:22.591653    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdac6d65-9077-4012-bcfb-b19556f35d5f-metrics-tls\") pod \"dns-default-gdlqf\" (UID: \"fdac6d65-9077-4012-bcfb-b19556f35d5f\") " pod="openshift-dns/dns-default-gdlqf"
Jul 17 09:27:22.591828 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:22.591718    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-registry-tls\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:22.591828 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:22.591797    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 09:27:22.591828 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:22.591809    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 09:27:22.591828 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:22.591826    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75c884f5f7-7k78v: secret "image-registry-tls" not found
Jul 17 09:27:22.591948 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:22.591849    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdac6d65-9077-4012-bcfb-b19556f35d5f-metrics-tls podName:fdac6d65-9077-4012-bcfb-b19556f35d5f nodeName:}" failed. No retries permitted until 2026-07-17 09:27:26.591836519 +0000 UTC m=+44.174474877 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fdac6d65-9077-4012-bcfb-b19556f35d5f-metrics-tls") pod "dns-default-gdlqf" (UID: "fdac6d65-9077-4012-bcfb-b19556f35d5f") : secret "dns-default-metrics-tls" not found
Jul 17 09:27:22.591948 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:22.591866    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-registry-tls podName:27c6d907-d449-4202-8700-dc369dadb9f4 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:26.591854987 +0000 UTC m=+44.174493341 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-registry-tls") pod "image-registry-75c884f5f7-7k78v" (UID: "27c6d907-d449-4202-8700-dc369dadb9f4") : secret "image-registry-tls" not found
Jul 17 09:27:23.081280 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:23.081224    2574 generic.go:356] "Generic (PLEG): container finished" podID="5c165f52-605b-48e9-9cff-bf1e0b7b97af" containerID="e740b44f949327d5765944d005af26dd8336683df23c393e942e0262090e77f2" exitCode=0
Jul 17 09:27:23.081672 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:23.081307    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d4ksz" event={"ID":"5c165f52-605b-48e9-9cff-bf1e0b7b97af","Type":"ContainerDied","Data":"e740b44f949327d5765944d005af26dd8336683df23c393e942e0262090e77f2"}
Jul 17 09:27:24.087281 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:24.087255    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-d4ksz" event={"ID":"5c165f52-605b-48e9-9cff-bf1e0b7b97af","Type":"ContainerStarted","Data":"13e8e79930cf1f6fc3b9a32a587da08b69f2cfe4ce610fc0ca21fc0d7e512995"}
Jul 17 09:27:24.111629 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:24.111591    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-d4ksz" podStartSLOduration=8.076074996 podStartE2EDuration="41.111576695s" totalImagesPullingTime="33.035501699s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:26:43 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:24.111087139 +0000 UTC m=+41.693725515" watchObservedRunningTime="2026-07-17 09:27:24.111576695 +0000 UTC m=+41.694215078"
Jul 17 09:27:25.090088 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:25.090058    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vqvdn" event={"ID":"78c11134-f4c9-4e8a-b87c-3ea12f21e6d1","Type":"ContainerStarted","Data":"0fa77dda8c08f1d2a8b565a7533224a6e2b47dafb3c20a24f25536bbaac3de56"}
Jul 17 09:27:25.105020 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:25.104981    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-vqvdn" podStartSLOduration=2.307819324 podStartE2EDuration="7.104965414s" totalImagesPullingTime="4.79714609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:27:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:25.104398749 +0000 UTC m=+42.687037125" watchObservedRunningTime="2026-07-17 09:27:25.104965414 +0000 UTC m=+42.687603789"
Jul 17 09:27:26.620790 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:26.620758    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdac6d65-9077-4012-bcfb-b19556f35d5f-metrics-tls\") pod \"dns-default-gdlqf\" (UID: \"fdac6d65-9077-4012-bcfb-b19556f35d5f\") " pod="openshift-dns/dns-default-gdlqf"
Jul 17 09:27:26.621179 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:26.620814    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-registry-tls\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:26.621179 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:26.620897    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 09:27:26.621179 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:26.620907    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75c884f5f7-7k78v: secret "image-registry-tls" not found
Jul 17 09:27:26.621179 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:26.620909    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 09:27:26.621179 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:26.620948    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-registry-tls podName:27c6d907-d449-4202-8700-dc369dadb9f4 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:34.620935492 +0000 UTC m=+52.203573848 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-registry-tls") pod "image-registry-75c884f5f7-7k78v" (UID: "27c6d907-d449-4202-8700-dc369dadb9f4") : secret "image-registry-tls" not found
Jul 17 09:27:26.621179 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:26.620983    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdac6d65-9077-4012-bcfb-b19556f35d5f-metrics-tls podName:fdac6d65-9077-4012-bcfb-b19556f35d5f nodeName:}" failed. No retries permitted until 2026-07-17 09:27:34.620966881 +0000 UTC m=+52.203605234 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fdac6d65-9077-4012-bcfb-b19556f35d5f-metrics-tls") pod "dns-default-gdlqf" (UID: "fdac6d65-9077-4012-bcfb-b19556f35d5f") : secret "dns-default-metrics-tls" not found
Jul 17 09:27:34.675422 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:34.675389    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdac6d65-9077-4012-bcfb-b19556f35d5f-metrics-tls\") pod \"dns-default-gdlqf\" (UID: \"fdac6d65-9077-4012-bcfb-b19556f35d5f\") " pod="openshift-dns/dns-default-gdlqf"
Jul 17 09:27:34.675893 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:34.675461    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-registry-tls\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:34.675893 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:34.675537    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 09:27:34.675893 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:34.675584    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 09:27:34.675893 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:34.675597    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75c884f5f7-7k78v: secret "image-registry-tls" not found
Jul 17 09:27:34.675893 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:34.675606    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdac6d65-9077-4012-bcfb-b19556f35d5f-metrics-tls podName:fdac6d65-9077-4012-bcfb-b19556f35d5f nodeName:}" failed. No retries permitted until 2026-07-17 09:27:50.675588128 +0000 UTC m=+68.258226482 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fdac6d65-9077-4012-bcfb-b19556f35d5f-metrics-tls") pod "dns-default-gdlqf" (UID: "fdac6d65-9077-4012-bcfb-b19556f35d5f") : secret "dns-default-metrics-tls" not found
Jul 17 09:27:34.675893 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:34.675641    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-registry-tls podName:27c6d907-d449-4202-8700-dc369dadb9f4 nodeName:}" failed. No retries permitted until 2026-07-17 09:27:50.675626488 +0000 UTC m=+68.258264858 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-registry-tls") pod "image-registry-75c884f5f7-7k78v" (UID: "27c6d907-d449-4202-8700-dc369dadb9f4") : secret "image-registry-tls" not found
Jul 17 09:27:34.983869 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:34.983810    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7ndbm"]
Jul 17 09:27:35.017558 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.013427    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jxtbz"]
Jul 17 09:27:35.017558 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.014347    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7ndbm"
Jul 17 09:27:35.023594 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.022970    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-f2kzt\""
Jul 17 09:27:35.023594 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.023183    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 17 09:27:35.023594 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.023438    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 17 09:27:35.025275 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.024061    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 17 09:27:35.028967 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.028948    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7ndbm"]
Jul 17 09:27:35.029082 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.028995    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jxtbz"]
Jul 17 09:27:35.029124 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.029084    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jxtbz"
Jul 17 09:27:35.031777 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.031757    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 17 09:27:35.031777 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.031774    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 17 09:27:35.031921 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.031780    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-82rbh\""
Jul 17 09:27:35.031921 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.031780    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 17 09:27:35.032278 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.032264    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 17 09:27:35.178532 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.178509    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgchv\" (UniqueName: \"kubernetes.io/projected/1931d80b-6799-4456-a479-23d203f2070a-kube-api-access-pgchv\") pod \"kube-storage-version-migrator-operator-7658d9b749-jxtbz\" (UID: \"1931d80b-6799-4456-a479-23d203f2070a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jxtbz"
Jul 17 09:27:35.178622 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.178545    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1931d80b-6799-4456-a479-23d203f2070a-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-jxtbz\" (UID: \"1931d80b-6799-4456-a479-23d203f2070a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jxtbz"
Jul 17 09:27:35.178622 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.178575    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/13d382d6-6740-4699-8438-15a9875eaf3c-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7ndbm\" (UID: \"13d382d6-6740-4699-8438-15a9875eaf3c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7ndbm"
Jul 17 09:27:35.178697 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.178625    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8jtq\" (UniqueName: \"kubernetes.io/projected/13d382d6-6740-4699-8438-15a9875eaf3c-kube-api-access-p8jtq\") pod \"cluster-samples-operator-79b7c55f87-7ndbm\" (UID: \"13d382d6-6740-4699-8438-15a9875eaf3c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7ndbm"
Jul 17 09:27:35.178697 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.178677    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1931d80b-6799-4456-a479-23d203f2070a-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-jxtbz\" (UID: \"1931d80b-6799-4456-a479-23d203f2070a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jxtbz"
Jul 17 09:27:35.279072 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.279027    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgchv\" (UniqueName: \"kubernetes.io/projected/1931d80b-6799-4456-a479-23d203f2070a-kube-api-access-pgchv\") pod \"kube-storage-version-migrator-operator-7658d9b749-jxtbz\" (UID: \"1931d80b-6799-4456-a479-23d203f2070a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jxtbz"
Jul 17 09:27:35.279072 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.279063    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1931d80b-6799-4456-a479-23d203f2070a-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-jxtbz\" (UID: \"1931d80b-6799-4456-a479-23d203f2070a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jxtbz"
Jul 17 09:27:35.279316 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.279099    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/13d382d6-6740-4699-8438-15a9875eaf3c-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7ndbm\" (UID: \"13d382d6-6740-4699-8438-15a9875eaf3c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7ndbm"
Jul 17 09:27:35.279316 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.279135    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8jtq\" (UniqueName: \"kubernetes.io/projected/13d382d6-6740-4699-8438-15a9875eaf3c-kube-api-access-p8jtq\") pod \"cluster-samples-operator-79b7c55f87-7ndbm\" (UID: \"13d382d6-6740-4699-8438-15a9875eaf3c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7ndbm"
Jul 17 09:27:35.279316 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.279202    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1931d80b-6799-4456-a479-23d203f2070a-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-jxtbz\" (UID: \"1931d80b-6799-4456-a479-23d203f2070a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jxtbz"
Jul 17 09:27:35.279316 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:35.279294    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 17 09:27:35.279534 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:35.279364    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13d382d6-6740-4699-8438-15a9875eaf3c-samples-operator-tls podName:13d382d6-6740-4699-8438-15a9875eaf3c nodeName:}" failed. No retries permitted until 2026-07-17 09:27:35.779344653 +0000 UTC m=+53.361983010 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/13d382d6-6740-4699-8438-15a9875eaf3c-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-7ndbm" (UID: "13d382d6-6740-4699-8438-15a9875eaf3c") : secret "samples-operator-tls" not found
Jul 17 09:27:35.279723 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.279706    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1931d80b-6799-4456-a479-23d203f2070a-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-jxtbz\" (UID: \"1931d80b-6799-4456-a479-23d203f2070a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jxtbz"
Jul 17 09:27:35.282582 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.282565    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1931d80b-6799-4456-a479-23d203f2070a-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-jxtbz\" (UID: \"1931d80b-6799-4456-a479-23d203f2070a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jxtbz"
Jul 17 09:27:35.289496 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.289475    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgchv\" (UniqueName: \"kubernetes.io/projected/1931d80b-6799-4456-a479-23d203f2070a-kube-api-access-pgchv\") pod \"kube-storage-version-migrator-operator-7658d9b749-jxtbz\" (UID: \"1931d80b-6799-4456-a479-23d203f2070a\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jxtbz"
Jul 17 09:27:35.290786 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.290766    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8jtq\" (UniqueName: \"kubernetes.io/projected/13d382d6-6740-4699-8438-15a9875eaf3c-kube-api-access-p8jtq\") pod \"cluster-samples-operator-79b7c55f87-7ndbm\" (UID: \"13d382d6-6740-4699-8438-15a9875eaf3c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7ndbm"
Jul 17 09:27:35.338895 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.338873    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jxtbz"
Jul 17 09:27:35.473553 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.473525    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jxtbz"]
Jul 17 09:27:35.477016 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:27:35.476986    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1931d80b_6799_4456_a479_23d203f2070a.slice/crio-f12a0238078c532dd11637fdeeff073927e4787181853a9bb89e6eb14fbcdc0c WatchSource:0}: Error finding container f12a0238078c532dd11637fdeeff073927e4787181853a9bb89e6eb14fbcdc0c: Status 404 returned error can't find the container with id f12a0238078c532dd11637fdeeff073927e4787181853a9bb89e6eb14fbcdc0c
Jul 17 09:27:35.783680 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:35.783649    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/13d382d6-6740-4699-8438-15a9875eaf3c-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7ndbm\" (UID: \"13d382d6-6740-4699-8438-15a9875eaf3c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7ndbm"
Jul 17 09:27:35.784038 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:35.783768    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 17 09:27:35.784038 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:35.783838    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13d382d6-6740-4699-8438-15a9875eaf3c-samples-operator-tls podName:13d382d6-6740-4699-8438-15a9875eaf3c nodeName:}" failed. No retries permitted until 2026-07-17 09:27:36.783817743 +0000 UTC m=+54.366456097 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/13d382d6-6740-4699-8438-15a9875eaf3c-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-7ndbm" (UID: "13d382d6-6740-4699-8438-15a9875eaf3c") : secret "samples-operator-tls" not found
Jul 17 09:27:36.113819 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:36.113758    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jxtbz" event={"ID":"1931d80b-6799-4456-a479-23d203f2070a","Type":"ContainerStarted","Data":"f12a0238078c532dd11637fdeeff073927e4787181853a9bb89e6eb14fbcdc0c"}
Jul 17 09:27:36.791728 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:36.791697    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/13d382d6-6740-4699-8438-15a9875eaf3c-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7ndbm\" (UID: \"13d382d6-6740-4699-8438-15a9875eaf3c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7ndbm"
Jul 17 09:27:36.792078 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:36.791840    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 17 09:27:36.792078 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:36.791898    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13d382d6-6740-4699-8438-15a9875eaf3c-samples-operator-tls podName:13d382d6-6740-4699-8438-15a9875eaf3c nodeName:}" failed. No retries permitted until 2026-07-17 09:27:38.791881939 +0000 UTC m=+56.374520296 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/13d382d6-6740-4699-8438-15a9875eaf3c-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-7ndbm" (UID: "13d382d6-6740-4699-8438-15a9875eaf3c") : secret "samples-operator-tls" not found
Jul 17 09:27:38.120534 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:38.120498    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jxtbz" event={"ID":"1931d80b-6799-4456-a479-23d203f2070a","Type":"ContainerStarted","Data":"a0ac9d069c9cad7b5bdfbc4a678ab6abd77501558eae3e488d8dde9f4c6ae56f"}
Jul 17 09:27:38.136162 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:38.136118    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jxtbz" podStartSLOduration=2.02539504 podStartE2EDuration="4.136103361s" totalImagesPullingTime="2.110708321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:27:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:38.135100788 +0000 UTC m=+55.717739175" watchObservedRunningTime="2026-07-17 09:27:38.136103361 +0000 UTC m=+55.718741736"
Jul 17 09:27:38.806546 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:38.806517    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/13d382d6-6740-4699-8438-15a9875eaf3c-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7ndbm\" (UID: \"13d382d6-6740-4699-8438-15a9875eaf3c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7ndbm"
Jul 17 09:27:38.806681 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:38.806648    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 17 09:27:38.806725 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:38.806703    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13d382d6-6740-4699-8438-15a9875eaf3c-samples-operator-tls podName:13d382d6-6740-4699-8438-15a9875eaf3c nodeName:}" failed. No retries permitted until 2026-07-17 09:27:42.806689836 +0000 UTC m=+60.389328193 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/13d382d6-6740-4699-8438-15a9875eaf3c-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-7ndbm" (UID: "13d382d6-6740-4699-8438-15a9875eaf3c") : secret "samples-operator-tls" not found
Jul 17 09:27:41.107989 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:41.107957    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zvktb"]
Jul 17 09:27:41.110005 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:41.109988    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zvktb"
Jul 17 09:27:41.112663 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:41.112641    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 17 09:27:41.113964 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:41.113947    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 17 09:27:41.114049 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:41.113966    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gmxxn\""
Jul 17 09:27:41.114049 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:41.114031    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 17 09:27:41.114167 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:41.113970    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 17 09:27:41.123678 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:41.120893    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zvktb"]
Jul 17 09:27:41.124840 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:41.124820    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7j98g\" (UniqueName: \"kubernetes.io/projected/44a47c07-93d4-4047-9e80-3ffa7f6b066e-kube-api-access-7j98g\") pod \"insights-runtime-extractor-zvktb\" (UID: \"44a47c07-93d4-4047-9e80-3ffa7f6b066e\") " pod="openshift-insights/insights-runtime-extractor-zvktb"
Jul 17 09:27:41.124934 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:41.124867    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/44a47c07-93d4-4047-9e80-3ffa7f6b066e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zvktb\" (UID: \"44a47c07-93d4-4047-9e80-3ffa7f6b066e\") " pod="openshift-insights/insights-runtime-extractor-zvktb"
Jul 17 09:27:41.124981 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:41.124953    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/44a47c07-93d4-4047-9e80-3ffa7f6b066e-data-volume\") pod \"insights-runtime-extractor-zvktb\" (UID: \"44a47c07-93d4-4047-9e80-3ffa7f6b066e\") " pod="openshift-insights/insights-runtime-extractor-zvktb"
Jul 17 09:27:41.125029 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:41.124984    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/44a47c07-93d4-4047-9e80-3ffa7f6b066e-crio-socket\") pod \"insights-runtime-extractor-zvktb\" (UID: \"44a47c07-93d4-4047-9e80-3ffa7f6b066e\") " pod="openshift-insights/insights-runtime-extractor-zvktb"
Jul 17 09:27:41.125063 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:41.125049    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/44a47c07-93d4-4047-9e80-3ffa7f6b066e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zvktb\" (UID: \"44a47c07-93d4-4047-9e80-3ffa7f6b066e\") " pod="openshift-insights/insights-runtime-extractor-zvktb"
Jul 17 09:27:41.225911 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:41.225884    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/44a47c07-93d4-4047-9e80-3ffa7f6b066e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zvktb\" (UID: \"44a47c07-93d4-4047-9e80-3ffa7f6b066e\") " pod="openshift-insights/insights-runtime-extractor-zvktb"
Jul 17 09:27:41.226008 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:41.225919    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7j98g\" (UniqueName: \"kubernetes.io/projected/44a47c07-93d4-4047-9e80-3ffa7f6b066e-kube-api-access-7j98g\") pod \"insights-runtime-extractor-zvktb\" (UID: \"44a47c07-93d4-4047-9e80-3ffa7f6b066e\") " pod="openshift-insights/insights-runtime-extractor-zvktb"
Jul 17 09:27:41.226008 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:41.225944    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/44a47c07-93d4-4047-9e80-3ffa7f6b066e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zvktb\" (UID: \"44a47c07-93d4-4047-9e80-3ffa7f6b066e\") " pod="openshift-insights/insights-runtime-extractor-zvktb"
Jul 17 09:27:41.226008 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:41.226000    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 09:27:41.226154 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:41.226026    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 09:27:41.226154 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:41.226056    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/44a47c07-93d4-4047-9e80-3ffa7f6b066e-kube-rbac-proxy-cm podName:44a47c07-93d4-4047-9e80-3ffa7f6b066e nodeName:}" failed. No retries permitted until 2026-07-17 09:27:41.726041856 +0000 UTC m=+59.308680209 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/44a47c07-93d4-4047-9e80-3ffa7f6b066e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zvktb" (UID: "44a47c07-93d4-4047-9e80-3ffa7f6b066e") : configmap "kube-rbac-proxy" not found
Jul 17 09:27:41.226154 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:41.226085    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/44a47c07-93d4-4047-9e80-3ffa7f6b066e-data-volume\") pod \"insights-runtime-extractor-zvktb\" (UID: \"44a47c07-93d4-4047-9e80-3ffa7f6b066e\") " pod="openshift-insights/insights-runtime-extractor-zvktb"
Jul 17 09:27:41.226154 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:41.226138    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/44a47c07-93d4-4047-9e80-3ffa7f6b066e-crio-socket\") pod \"insights-runtime-extractor-zvktb\" (UID: \"44a47c07-93d4-4047-9e80-3ffa7f6b066e\") " pod="openshift-insights/insights-runtime-extractor-zvktb"
Jul 17 09:27:41.226348 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:41.226158    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44a47c07-93d4-4047-9e80-3ffa7f6b066e-insights-runtime-extractor-tls podName:44a47c07-93d4-4047-9e80-3ffa7f6b066e nodeName:}" failed. No retries permitted until 2026-07-17 09:27:41.726149333 +0000 UTC m=+59.308787687 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/44a47c07-93d4-4047-9e80-3ffa7f6b066e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zvktb" (UID: "44a47c07-93d4-4047-9e80-3ffa7f6b066e") : secret "insights-runtime-extractor-tls" not found
Jul 17 09:27:41.226348 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:41.226208    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/44a47c07-93d4-4047-9e80-3ffa7f6b066e-crio-socket\") pod \"insights-runtime-extractor-zvktb\" (UID: \"44a47c07-93d4-4047-9e80-3ffa7f6b066e\") " pod="openshift-insights/insights-runtime-extractor-zvktb"
Jul 17 09:27:41.226348 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:41.226341    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/44a47c07-93d4-4047-9e80-3ffa7f6b066e-data-volume\") pod \"insights-runtime-extractor-zvktb\" (UID: \"44a47c07-93d4-4047-9e80-3ffa7f6b066e\") " pod="openshift-insights/insights-runtime-extractor-zvktb"
Jul 17 09:27:41.234327 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:41.234311    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7j98g\" (UniqueName: \"kubernetes.io/projected/44a47c07-93d4-4047-9e80-3ffa7f6b066e-kube-api-access-7j98g\") pod \"insights-runtime-extractor-zvktb\" (UID: \"44a47c07-93d4-4047-9e80-3ffa7f6b066e\") " pod="openshift-insights/insights-runtime-extractor-zvktb"
Jul 17 09:27:41.730137 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:41.730106    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/44a47c07-93d4-4047-9e80-3ffa7f6b066e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zvktb\" (UID: \"44a47c07-93d4-4047-9e80-3ffa7f6b066e\") " pod="openshift-insights/insights-runtime-extractor-zvktb"
Jul 17 09:27:41.730261 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:41.730155    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/44a47c07-93d4-4047-9e80-3ffa7f6b066e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zvktb\" (UID: \"44a47c07-93d4-4047-9e80-3ffa7f6b066e\") " pod="openshift-insights/insights-runtime-extractor-zvktb"
Jul 17 09:27:41.730261 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:41.730211    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 09:27:41.730350 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:41.730267    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 09:27:41.730350 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:41.730278    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/44a47c07-93d4-4047-9e80-3ffa7f6b066e-kube-rbac-proxy-cm podName:44a47c07-93d4-4047-9e80-3ffa7f6b066e nodeName:}" failed. No retries permitted until 2026-07-17 09:27:42.730263775 +0000 UTC m=+60.312902133 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/44a47c07-93d4-4047-9e80-3ffa7f6b066e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zvktb" (UID: "44a47c07-93d4-4047-9e80-3ffa7f6b066e") : configmap "kube-rbac-proxy" not found
Jul 17 09:27:41.730350 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:41.730301    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44a47c07-93d4-4047-9e80-3ffa7f6b066e-insights-runtime-extractor-tls podName:44a47c07-93d4-4047-9e80-3ffa7f6b066e nodeName:}" failed. No retries permitted until 2026-07-17 09:27:42.730291164 +0000 UTC m=+60.312929518 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/44a47c07-93d4-4047-9e80-3ffa7f6b066e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zvktb" (UID: "44a47c07-93d4-4047-9e80-3ffa7f6b066e") : secret "insights-runtime-extractor-tls" not found
Jul 17 09:27:42.736849 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:42.736819    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/44a47c07-93d4-4047-9e80-3ffa7f6b066e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zvktb\" (UID: \"44a47c07-93d4-4047-9e80-3ffa7f6b066e\") " pod="openshift-insights/insights-runtime-extractor-zvktb"
Jul 17 09:27:42.737270 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:42.736942    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/44a47c07-93d4-4047-9e80-3ffa7f6b066e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zvktb\" (UID: \"44a47c07-93d4-4047-9e80-3ffa7f6b066e\") " pod="openshift-insights/insights-runtime-extractor-zvktb"
Jul 17 09:27:42.737270 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:42.736954    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 09:27:42.737270 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:42.737003    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44a47c07-93d4-4047-9e80-3ffa7f6b066e-insights-runtime-extractor-tls podName:44a47c07-93d4-4047-9e80-3ffa7f6b066e nodeName:}" failed. No retries permitted until 2026-07-17 09:27:44.7369908 +0000 UTC m=+62.319629153 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/44a47c07-93d4-4047-9e80-3ffa7f6b066e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zvktb" (UID: "44a47c07-93d4-4047-9e80-3ffa7f6b066e") : secret "insights-runtime-extractor-tls" not found
Jul 17 09:27:42.737270 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:42.737030    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 09:27:42.737270 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:42.737078    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/44a47c07-93d4-4047-9e80-3ffa7f6b066e-kube-rbac-proxy-cm podName:44a47c07-93d4-4047-9e80-3ffa7f6b066e nodeName:}" failed. No retries permitted until 2026-07-17 09:27:44.737062968 +0000 UTC m=+62.319701324 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/44a47c07-93d4-4047-9e80-3ffa7f6b066e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zvktb" (UID: "44a47c07-93d4-4047-9e80-3ffa7f6b066e") : configmap "kube-rbac-proxy" not found
Jul 17 09:27:42.837736 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:42.837712    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/13d382d6-6740-4699-8438-15a9875eaf3c-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7ndbm\" (UID: \"13d382d6-6740-4699-8438-15a9875eaf3c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7ndbm"
Jul 17 09:27:42.837865 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:42.837827    2574 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 17 09:27:42.837907 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:42.837867    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13d382d6-6740-4699-8438-15a9875eaf3c-samples-operator-tls podName:13d382d6-6740-4699-8438-15a9875eaf3c nodeName:}" failed. No retries permitted until 2026-07-17 09:27:50.837857251 +0000 UTC m=+68.420495605 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/13d382d6-6740-4699-8438-15a9875eaf3c-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-7ndbm" (UID: "13d382d6-6740-4699-8438-15a9875eaf3c") : secret "samples-operator-tls" not found
Jul 17 09:27:43.635001 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:43.634978    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fz9bt_eccc078f-2e4c-4a45-9056-d0cd7053faf6/dns-node-resolver/0.log"
Jul 17 09:27:44.438178 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:44.438154    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4x7kz_abec7743-4f1a-45ad-bddf-3f9c0b7d42dc/node-ca/0.log"
Jul 17 09:27:44.752513 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:44.752457    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/44a47c07-93d4-4047-9e80-3ffa7f6b066e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zvktb\" (UID: \"44a47c07-93d4-4047-9e80-3ffa7f6b066e\") " pod="openshift-insights/insights-runtime-extractor-zvktb"
Jul 17 09:27:44.752513 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:44.752499    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/44a47c07-93d4-4047-9e80-3ffa7f6b066e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zvktb\" (UID: \"44a47c07-93d4-4047-9e80-3ffa7f6b066e\") " pod="openshift-insights/insights-runtime-extractor-zvktb"
Jul 17 09:27:44.752683 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:44.752591    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 09:27:44.752683 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:44.752597    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 09:27:44.752683 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:44.752645    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44a47c07-93d4-4047-9e80-3ffa7f6b066e-insights-runtime-extractor-tls podName:44a47c07-93d4-4047-9e80-3ffa7f6b066e nodeName:}" failed. No retries permitted until 2026-07-17 09:27:48.752633873 +0000 UTC m=+66.335272231 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/44a47c07-93d4-4047-9e80-3ffa7f6b066e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zvktb" (UID: "44a47c07-93d4-4047-9e80-3ffa7f6b066e") : secret "insights-runtime-extractor-tls" not found
Jul 17 09:27:44.752683 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:44.752679    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/44a47c07-93d4-4047-9e80-3ffa7f6b066e-kube-rbac-proxy-cm podName:44a47c07-93d4-4047-9e80-3ffa7f6b066e nodeName:}" failed. No retries permitted until 2026-07-17 09:27:48.752672529 +0000 UTC m=+66.335310882 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/44a47c07-93d4-4047-9e80-3ffa7f6b066e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zvktb" (UID: "44a47c07-93d4-4047-9e80-3ffa7f6b066e") : configmap "kube-rbac-proxy" not found
Jul 17 09:27:45.061606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:45.061550    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dh564"
Jul 17 09:27:45.636401 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:45.636377    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vqvdn_78c11134-f4c9-4e8a-b87c-3ea12f21e6d1/serve-healthcheck-canary/0.log"
Jul 17 09:27:46.237288 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:46.237257    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-jxtbz_1931d80b-6799-4456-a479-23d203f2070a/kube-storage-version-migrator-operator/0.log"
Jul 17 09:27:48.580679 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:48.580650    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb87a089-9075-4b84-a8b6-98cfaa1d9387-metrics-certs\") pod \"network-metrics-daemon-s62sx\" (UID: \"fb87a089-9075-4b84-a8b6-98cfaa1d9387\") " pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:27:48.583626 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:48.583607    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 17 09:27:48.591137 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:48.591120    2574 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 17 09:27:48.591207 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:48.591166    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb87a089-9075-4b84-a8b6-98cfaa1d9387-metrics-certs podName:fb87a089-9075-4b84-a8b6-98cfaa1d9387 nodeName:}" failed. No retries permitted until 2026-07-17 09:28:52.591153036 +0000 UTC m=+130.173791389 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/fb87a089-9075-4b84-a8b6-98cfaa1d9387-metrics-certs") pod "network-metrics-daemon-s62sx" (UID: "fb87a089-9075-4b84-a8b6-98cfaa1d9387") : secret "metrics-daemon-secret" not found
Jul 17 09:27:48.681202 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:48.681170    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f357001d-cac3-4c7d-b427-7cc231e3d4ff-original-pull-secret\") pod \"global-pull-secret-syncer-trvgv\" (UID: \"f357001d-cac3-4c7d-b427-7cc231e3d4ff\") " pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:27:48.681301 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:48.681290    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf448\" (UniqueName: \"kubernetes.io/projected/ba08cebc-2cf5-4174-9698-031b279f9258-kube-api-access-cf448\") pod \"network-check-target-qks8f\" (UID: \"ba08cebc-2cf5-4174-9698-031b279f9258\") " pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:27:48.684357 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:48.684339    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 17 09:27:48.684575 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:48.684560    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 17 09:27:48.693790 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:48.693755    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f357001d-cac3-4c7d-b427-7cc231e3d4ff-original-pull-secret\") pod \"global-pull-secret-syncer-trvgv\" (UID: \"f357001d-cac3-4c7d-b427-7cc231e3d4ff\") " pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:27:48.693968 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:48.693956    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 17 09:27:48.704388 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:48.704372    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf448\" (UniqueName: \"kubernetes.io/projected/ba08cebc-2cf5-4174-9698-031b279f9258-kube-api-access-cf448\") pod \"network-check-target-qks8f\" (UID: \"ba08cebc-2cf5-4174-9698-031b279f9258\") " pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:27:48.781791 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:48.781768    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/44a47c07-93d4-4047-9e80-3ffa7f6b066e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zvktb\" (UID: \"44a47c07-93d4-4047-9e80-3ffa7f6b066e\") " pod="openshift-insights/insights-runtime-extractor-zvktb"
Jul 17 09:27:48.781871 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:48.781806    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/44a47c07-93d4-4047-9e80-3ffa7f6b066e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zvktb\" (UID: \"44a47c07-93d4-4047-9e80-3ffa7f6b066e\") " pod="openshift-insights/insights-runtime-extractor-zvktb"
Jul 17 09:27:48.781912 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:48.781887    2574 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 17 09:27:48.781912 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:48.781903    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 09:27:48.782006 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:48.781941    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/44a47c07-93d4-4047-9e80-3ffa7f6b066e-kube-rbac-proxy-cm podName:44a47c07-93d4-4047-9e80-3ffa7f6b066e nodeName:}" failed. No retries permitted until 2026-07-17 09:27:56.781926045 +0000 UTC m=+74.364564407 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/44a47c07-93d4-4047-9e80-3ffa7f6b066e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zvktb" (UID: "44a47c07-93d4-4047-9e80-3ffa7f6b066e") : configmap "kube-rbac-proxy" not found
Jul 17 09:27:48.782006 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:48.781955    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44a47c07-93d4-4047-9e80-3ffa7f6b066e-insights-runtime-extractor-tls podName:44a47c07-93d4-4047-9e80-3ffa7f6b066e nodeName:}" failed. No retries permitted until 2026-07-17 09:27:56.781949016 +0000 UTC m=+74.364587370 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/44a47c07-93d4-4047-9e80-3ffa7f6b066e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zvktb" (UID: "44a47c07-93d4-4047-9e80-3ffa7f6b066e") : secret "insights-runtime-extractor-tls" not found
Jul 17 09:27:48.902518 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:48.902474    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-trvgv"
Jul 17 09:27:48.911811 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:48.911791    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qg8vm\""
Jul 17 09:27:48.919780 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:48.919762    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:27:49.033176 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:49.033037    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-trvgv"]
Jul 17 09:27:49.035748 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:27:49.035719    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf357001d_cac3_4c7d_b427_7cc231e3d4ff.slice/crio-9618937c57a7e3bbd1681342dc0279d8027644e927ac401be7ad1fda64e8765a WatchSource:0}: Error finding container 9618937c57a7e3bbd1681342dc0279d8027644e927ac401be7ad1fda64e8765a: Status 404 returned error can't find the container with id 9618937c57a7e3bbd1681342dc0279d8027644e927ac401be7ad1fda64e8765a
Jul 17 09:27:49.051711 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:49.051693    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qks8f"]
Jul 17 09:27:49.054030 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:27:49.054010    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba08cebc_2cf5_4174_9698_031b279f9258.slice/crio-7e93f9cafa000b50dfb49c3794d9439d44d3f6e10d1a1be06634da7291cedec4 WatchSource:0}: Error finding container 7e93f9cafa000b50dfb49c3794d9439d44d3f6e10d1a1be06634da7291cedec4: Status 404 returned error can't find the container with id 7e93f9cafa000b50dfb49c3794d9439d44d3f6e10d1a1be06634da7291cedec4
Jul 17 09:27:49.146516 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:49.146487    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qks8f" event={"ID":"ba08cebc-2cf5-4174-9698-031b279f9258","Type":"ContainerStarted","Data":"7e93f9cafa000b50dfb49c3794d9439d44d3f6e10d1a1be06634da7291cedec4"}
Jul 17 09:27:49.147457 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:49.147439    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-trvgv" event={"ID":"f357001d-cac3-4c7d-b427-7cc231e3d4ff","Type":"ContainerStarted","Data":"9618937c57a7e3bbd1681342dc0279d8027644e927ac401be7ad1fda64e8765a"}
Jul 17 09:27:50.697176 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:50.697140    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdac6d65-9077-4012-bcfb-b19556f35d5f-metrics-tls\") pod \"dns-default-gdlqf\" (UID: \"fdac6d65-9077-4012-bcfb-b19556f35d5f\") " pod="openshift-dns/dns-default-gdlqf"
Jul 17 09:27:50.697614 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:50.697254    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-registry-tls\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:27:50.697614 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:50.697328    2574 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 17 09:27:50.697614 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:50.697365    2574 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 17 09:27:50.697614 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:50.697380    2574 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-75c884f5f7-7k78v: secret "image-registry-tls" not found
Jul 17 09:27:50.697614 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:50.697405    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdac6d65-9077-4012-bcfb-b19556f35d5f-metrics-tls podName:fdac6d65-9077-4012-bcfb-b19556f35d5f nodeName:}" failed. No retries permitted until 2026-07-17 09:28:22.697386974 +0000 UTC m=+100.280025344 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fdac6d65-9077-4012-bcfb-b19556f35d5f-metrics-tls") pod "dns-default-gdlqf" (UID: "fdac6d65-9077-4012-bcfb-b19556f35d5f") : secret "dns-default-metrics-tls" not found
Jul 17 09:27:50.697614 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:50.697429    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-registry-tls podName:27c6d907-d449-4202-8700-dc369dadb9f4 nodeName:}" failed. No retries permitted until 2026-07-17 09:28:22.697413418 +0000 UTC m=+100.280051776 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-registry-tls") pod "image-registry-75c884f5f7-7k78v" (UID: "27c6d907-d449-4202-8700-dc369dadb9f4") : secret "image-registry-tls" not found
Jul 17 09:27:50.898442 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:50.898408    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/13d382d6-6740-4699-8438-15a9875eaf3c-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7ndbm\" (UID: \"13d382d6-6740-4699-8438-15a9875eaf3c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7ndbm"
Jul 17 09:27:50.900942 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:50.900915    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/13d382d6-6740-4699-8438-15a9875eaf3c-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-7ndbm\" (UID: \"13d382d6-6740-4699-8438-15a9875eaf3c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7ndbm"
Jul 17 09:27:50.936545 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:50.936519    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-f2kzt\""
Jul 17 09:27:50.944917 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:50.944886    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7ndbm"
Jul 17 09:27:51.090160 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:51.090124    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7ndbm"]
Jul 17 09:27:53.159488 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:53.159455    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7ndbm" event={"ID":"13d382d6-6740-4699-8438-15a9875eaf3c","Type":"ContainerStarted","Data":"73c5a15fe0d34da5da35fa3e528b2786b70f253f85d56b76841632bc62d308ac"}
Jul 17 09:27:54.164441 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:54.164401    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qks8f" event={"ID":"ba08cebc-2cf5-4174-9698-031b279f9258","Type":"ContainerStarted","Data":"8ce73d1cebe92836f0f53a868114683b09c11fe4d44a08c1da507e2f14a9e808"}
Jul 17 09:27:54.164899 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:54.164594    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:27:54.166058 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:54.166033    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-trvgv" event={"ID":"f357001d-cac3-4c7d-b427-7cc231e3d4ff","Type":"ContainerStarted","Data":"a32981408b28c1ab548ab9e7c46df168c23cf44e70455f3d7fd90a9f76a27eab"}
Jul 17 09:27:54.179303 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:54.179260    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qks8f" podStartSLOduration=66.601033864 podStartE2EDuration="1m11.179228325s" totalImagesPullingTime="4.578194461s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:26:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:54.178394337 +0000 UTC m=+71.761032716" watchObservedRunningTime="2026-07-17 09:27:54.179228325 +0000 UTC m=+71.761866682"
Jul 17 09:27:54.191998 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:54.191950    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-trvgv" podStartSLOduration=66.591102761 podStartE2EDuration="1m11.191932966s" totalImagesPullingTime="4.600830205s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:26:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:54.19114694 +0000 UTC m=+71.773785318" watchObservedRunningTime="2026-07-17 09:27:54.191932966 +0000 UTC m=+71.774571343"
Jul 17 09:27:55.170425 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:55.170392    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7ndbm" event={"ID":"13d382d6-6740-4699-8438-15a9875eaf3c","Type":"ContainerStarted","Data":"7ee0ff7b9fa1b3f63beed0a4ecc17a85a6a782fa448ec4f60150edb5bc3da97d"}
Jul 17 09:27:55.170816 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:55.170455    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7ndbm" event={"ID":"13d382d6-6740-4699-8438-15a9875eaf3c","Type":"ContainerStarted","Data":"356651eb488a934b60c3cc982aea85abd300f8c7dbe2fa83de947da199718b4e"}
Jul 17 09:27:55.185499 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:55.185464    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-7ndbm" podStartSLOduration=18.535079852 podStartE2EDuration="21.185446537s" totalImagesPullingTime="2.650366685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:27:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:27:55.185260182 +0000 UTC m=+72.767898555" watchObservedRunningTime="2026-07-17 09:27:55.185446537 +0000 UTC m=+72.768084915"
Jul 17 09:27:56.841710 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:56.841680    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/44a47c07-93d4-4047-9e80-3ffa7f6b066e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zvktb\" (UID: \"44a47c07-93d4-4047-9e80-3ffa7f6b066e\") " pod="openshift-insights/insights-runtime-extractor-zvktb"
Jul 17 09:27:56.842416 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:56.841747    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/44a47c07-93d4-4047-9e80-3ffa7f6b066e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zvktb\" (UID: \"44a47c07-93d4-4047-9e80-3ffa7f6b066e\") " pod="openshift-insights/insights-runtime-extractor-zvktb"
Jul 17 09:27:56.842416 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:56.841839    2574 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 17 09:27:56.842416 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:27:56.841910    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44a47c07-93d4-4047-9e80-3ffa7f6b066e-insights-runtime-extractor-tls podName:44a47c07-93d4-4047-9e80-3ffa7f6b066e nodeName:}" failed. No retries permitted until 2026-07-17 09:28:12.841891912 +0000 UTC m=+90.424530267 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/44a47c07-93d4-4047-9e80-3ffa7f6b066e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zvktb" (UID: "44a47c07-93d4-4047-9e80-3ffa7f6b066e") : secret "insights-runtime-extractor-tls" not found
Jul 17 09:27:56.842416 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:27:56.842143    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/44a47c07-93d4-4047-9e80-3ffa7f6b066e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zvktb\" (UID: \"44a47c07-93d4-4047-9e80-3ffa7f6b066e\") " pod="openshift-insights/insights-runtime-extractor-zvktb"
Jul 17 09:28:04.609145 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:04.609038    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fb86f68cd-wfjwg"]
Jul 17 09:28:04.610965 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:04.610945    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fb86f68cd-wfjwg"
Jul 17 09:28:04.615965 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:04.615946    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 17 09:28:04.616060 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:04.615974    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 17 09:28:04.627986 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:04.627968    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-kdv4p\""
Jul 17 09:28:04.628673 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:04.628652    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 17 09:28:04.628804 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:04.628789    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 17 09:28:04.640192 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:04.640169    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fb86f68cd-wfjwg"]
Jul 17 09:28:04.695483 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:04.695460    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/18e98531-04c8-4a42-aa8b-07476a4c63ca-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-fb86f68cd-wfjwg\" (UID: \"18e98531-04c8-4a42-aa8b-07476a4c63ca\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fb86f68cd-wfjwg"
Jul 17 09:28:04.695603 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:04.695583    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6mbl\" (UniqueName: \"kubernetes.io/projected/18e98531-04c8-4a42-aa8b-07476a4c63ca-kube-api-access-t6mbl\") pod \"managed-serviceaccount-addon-agent-fb86f68cd-wfjwg\" (UID: \"18e98531-04c8-4a42-aa8b-07476a4c63ca\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fb86f68cd-wfjwg"
Jul 17 09:28:04.704811 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:04.704788    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-hjfk5"]
Jul 17 09:28:04.706482 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:04.706464    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-hjfk5"
Jul 17 09:28:04.713356 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:04.713335    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 17 09:28:04.713449 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:04.713382    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-kpsk6\""
Jul 17 09:28:04.713499 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:04.713450    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 17 09:28:04.728020 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:04.728000    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-hjfk5"]
Jul 17 09:28:04.796717 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:04.796695    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6mbl\" (UniqueName: \"kubernetes.io/projected/18e98531-04c8-4a42-aa8b-07476a4c63ca-kube-api-access-t6mbl\") pod \"managed-serviceaccount-addon-agent-fb86f68cd-wfjwg\" (UID: \"18e98531-04c8-4a42-aa8b-07476a4c63ca\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fb86f68cd-wfjwg"
Jul 17 09:28:04.796849 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:04.796771    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/18e98531-04c8-4a42-aa8b-07476a4c63ca-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-fb86f68cd-wfjwg\" (UID: \"18e98531-04c8-4a42-aa8b-07476a4c63ca\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fb86f68cd-wfjwg"
Jul 17 09:28:04.796849 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:04.796810    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj6cx\" (UniqueName: \"kubernetes.io/projected/21d479b5-ee17-4fb5-acf9-541487651af1-kube-api-access-pj6cx\") pod \"downloads-5bc67dcd46-hjfk5\" (UID: \"21d479b5-ee17-4fb5-acf9-541487651af1\") " pod="openshift-console/downloads-5bc67dcd46-hjfk5"
Jul 17 09:28:04.799276 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:04.799257    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/18e98531-04c8-4a42-aa8b-07476a4c63ca-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-fb86f68cd-wfjwg\" (UID: \"18e98531-04c8-4a42-aa8b-07476a4c63ca\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fb86f68cd-wfjwg"
Jul 17 09:28:04.810349 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:04.810329    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6mbl\" (UniqueName: \"kubernetes.io/projected/18e98531-04c8-4a42-aa8b-07476a4c63ca-kube-api-access-t6mbl\") pod \"managed-serviceaccount-addon-agent-fb86f68cd-wfjwg\" (UID: \"18e98531-04c8-4a42-aa8b-07476a4c63ca\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fb86f68cd-wfjwg"
Jul 17 09:28:04.897412 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:04.897366    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj6cx\" (UniqueName: \"kubernetes.io/projected/21d479b5-ee17-4fb5-acf9-541487651af1-kube-api-access-pj6cx\") pod \"downloads-5bc67dcd46-hjfk5\" (UID: \"21d479b5-ee17-4fb5-acf9-541487651af1\") " pod="openshift-console/downloads-5bc67dcd46-hjfk5"
Jul 17 09:28:04.911758 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:04.911733    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj6cx\" (UniqueName: \"kubernetes.io/projected/21d479b5-ee17-4fb5-acf9-541487651af1-kube-api-access-pj6cx\") pod \"downloads-5bc67dcd46-hjfk5\" (UID: \"21d479b5-ee17-4fb5-acf9-541487651af1\") " pod="openshift-console/downloads-5bc67dcd46-hjfk5"
Jul 17 09:28:04.929840 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:04.929820    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fb86f68cd-wfjwg"
Jul 17 09:28:05.014762 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:05.014732    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-hjfk5"
Jul 17 09:28:05.051729 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:05.051702    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fb86f68cd-wfjwg"]
Jul 17 09:28:05.054082 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:28:05.054049    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18e98531_04c8_4a42_aa8b_07476a4c63ca.slice/crio-91ee033213f67a91e88010e712ac7cb521088397947e475d20d0424daabd5665 WatchSource:0}: Error finding container 91ee033213f67a91e88010e712ac7cb521088397947e475d20d0424daabd5665: Status 404 returned error can't find the container with id 91ee033213f67a91e88010e712ac7cb521088397947e475d20d0424daabd5665
Jul 17 09:28:05.170954 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:05.170927    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-hjfk5"]
Jul 17 09:28:05.174025 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:28:05.173998    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21d479b5_ee17_4fb5_acf9_541487651af1.slice/crio-c503de60c630a6541b898165199b369d33dfb7852dce85f10c4ceea33b78996c WatchSource:0}: Error finding container c503de60c630a6541b898165199b369d33dfb7852dce85f10c4ceea33b78996c: Status 404 returned error can't find the container with id c503de60c630a6541b898165199b369d33dfb7852dce85f10c4ceea33b78996c
Jul 17 09:28:05.198134 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:05.198107    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-hjfk5" event={"ID":"21d479b5-ee17-4fb5-acf9-541487651af1","Type":"ContainerStarted","Data":"c503de60c630a6541b898165199b369d33dfb7852dce85f10c4ceea33b78996c"}
Jul 17 09:28:05.199018 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:05.198998    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fb86f68cd-wfjwg" event={"ID":"18e98531-04c8-4a42-aa8b-07476a4c63ca","Type":"ContainerStarted","Data":"91ee033213f67a91e88010e712ac7cb521088397947e475d20d0424daabd5665"}
Jul 17 09:28:08.210988 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:08.210944    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fb86f68cd-wfjwg" event={"ID":"18e98531-04c8-4a42-aa8b-07476a4c63ca","Type":"ContainerStarted","Data":"ca7b738ba73c2cdd929b3655c5bb60ca26130c88748bae1a0130fc114e44ccc2"}
Jul 17 09:28:08.231575 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:08.231523    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-fb86f68cd-wfjwg" podStartSLOduration=1.901576925 podStartE2EDuration="4.231501291s" totalImagesPullingTime="2.329924366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:28:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:08.230534707 +0000 UTC m=+85.813173082" watchObservedRunningTime="2026-07-17 09:28:08.231501291 +0000 UTC m=+85.814139677"
Jul 17 09:28:09.089856 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.089825    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-689d766b54-g8vpn"]
Jul 17 09:28:09.092301 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.092272    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-689d766b54-g8vpn"
Jul 17 09:28:09.096392 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.096367    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-692fj\""
Jul 17 09:28:09.096515 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.096482    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 17 09:28:09.096575 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.096368    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 17 09:28:09.096828 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.096368    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 17 09:28:09.096828 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.096684    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 17 09:28:09.096828 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.096772    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 17 09:28:09.107042 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.107006    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-689d766b54-g8vpn"]
Jul 17 09:28:09.127197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.127173    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72wfd\" (UniqueName: \"kubernetes.io/projected/1b1d64eb-7388-44f0-8600-2dcbea6896c7-kube-api-access-72wfd\") pod \"console-689d766b54-g8vpn\" (UID: \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\") " pod="openshift-console/console-689d766b54-g8vpn"
Jul 17 09:28:09.127314 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.127219    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b1d64eb-7388-44f0-8600-2dcbea6896c7-console-config\") pod \"console-689d766b54-g8vpn\" (UID: \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\") " pod="openshift-console/console-689d766b54-g8vpn"
Jul 17 09:28:09.127314 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.127291    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b1d64eb-7388-44f0-8600-2dcbea6896c7-service-ca\") pod \"console-689d766b54-g8vpn\" (UID: \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\") " pod="openshift-console/console-689d766b54-g8vpn"
Jul 17 09:28:09.127425 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.127319    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b1d64eb-7388-44f0-8600-2dcbea6896c7-oauth-serving-cert\") pod \"console-689d766b54-g8vpn\" (UID: \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\") " pod="openshift-console/console-689d766b54-g8vpn"
Jul 17 09:28:09.127425 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.127390    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b1d64eb-7388-44f0-8600-2dcbea6896c7-console-serving-cert\") pod \"console-689d766b54-g8vpn\" (UID: \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\") " pod="openshift-console/console-689d766b54-g8vpn"
Jul 17 09:28:09.127425 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.127418    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b1d64eb-7388-44f0-8600-2dcbea6896c7-console-oauth-config\") pod \"console-689d766b54-g8vpn\" (UID: \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\") " pod="openshift-console/console-689d766b54-g8vpn"
Jul 17 09:28:09.227970 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.227940    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72wfd\" (UniqueName: \"kubernetes.io/projected/1b1d64eb-7388-44f0-8600-2dcbea6896c7-kube-api-access-72wfd\") pod \"console-689d766b54-g8vpn\" (UID: \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\") " pod="openshift-console/console-689d766b54-g8vpn"
Jul 17 09:28:09.228394 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.227982    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b1d64eb-7388-44f0-8600-2dcbea6896c7-console-config\") pod \"console-689d766b54-g8vpn\" (UID: \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\") " pod="openshift-console/console-689d766b54-g8vpn"
Jul 17 09:28:09.228394 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.228016    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b1d64eb-7388-44f0-8600-2dcbea6896c7-service-ca\") pod \"console-689d766b54-g8vpn\" (UID: \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\") " pod="openshift-console/console-689d766b54-g8vpn"
Jul 17 09:28:09.228394 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.228042    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b1d64eb-7388-44f0-8600-2dcbea6896c7-oauth-serving-cert\") pod \"console-689d766b54-g8vpn\" (UID: \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\") " pod="openshift-console/console-689d766b54-g8vpn"
Jul 17 09:28:09.228394 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.228092    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b1d64eb-7388-44f0-8600-2dcbea6896c7-console-serving-cert\") pod \"console-689d766b54-g8vpn\" (UID: \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\") " pod="openshift-console/console-689d766b54-g8vpn"
Jul 17 09:28:09.228394 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.228120    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b1d64eb-7388-44f0-8600-2dcbea6896c7-console-oauth-config\") pod \"console-689d766b54-g8vpn\" (UID: \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\") " pod="openshift-console/console-689d766b54-g8vpn"
Jul 17 09:28:09.228740 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.228717    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b1d64eb-7388-44f0-8600-2dcbea6896c7-console-config\") pod \"console-689d766b54-g8vpn\" (UID: \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\") " pod="openshift-console/console-689d766b54-g8vpn"
Jul 17 09:28:09.228921 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.228875    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b1d64eb-7388-44f0-8600-2dcbea6896c7-oauth-serving-cert\") pod \"console-689d766b54-g8vpn\" (UID: \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\") " pod="openshift-console/console-689d766b54-g8vpn"
Jul 17 09:28:09.229097 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.229049    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b1d64eb-7388-44f0-8600-2dcbea6896c7-service-ca\") pod \"console-689d766b54-g8vpn\" (UID: \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\") " pod="openshift-console/console-689d766b54-g8vpn"
Jul 17 09:28:09.231361 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.231337    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b1d64eb-7388-44f0-8600-2dcbea6896c7-console-oauth-config\") pod \"console-689d766b54-g8vpn\" (UID: \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\") " pod="openshift-console/console-689d766b54-g8vpn"
Jul 17 09:28:09.231465 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.231384    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b1d64eb-7388-44f0-8600-2dcbea6896c7-console-serving-cert\") pod \"console-689d766b54-g8vpn\" (UID: \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\") " pod="openshift-console/console-689d766b54-g8vpn"
Jul 17 09:28:09.238477 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.238458    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72wfd\" (UniqueName: \"kubernetes.io/projected/1b1d64eb-7388-44f0-8600-2dcbea6896c7-kube-api-access-72wfd\") pod \"console-689d766b54-g8vpn\" (UID: \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\") " pod="openshift-console/console-689d766b54-g8vpn"
Jul 17 09:28:09.403753 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.403670    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-689d766b54-g8vpn"
Jul 17 09:28:09.542441 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:09.542415    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-689d766b54-g8vpn"]
Jul 17 09:28:09.544915 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:28:09.544885    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b1d64eb_7388_44f0_8600_2dcbea6896c7.slice/crio-125bbe8ec24cd0e2628f02e2ff52129ee039666f3cd62c68a1b60bf2cea6f392 WatchSource:0}: Error finding container 125bbe8ec24cd0e2628f02e2ff52129ee039666f3cd62c68a1b60bf2cea6f392: Status 404 returned error can't find the container with id 125bbe8ec24cd0e2628f02e2ff52129ee039666f3cd62c68a1b60bf2cea6f392
Jul 17 09:28:10.219491 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:10.219455    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-689d766b54-g8vpn" event={"ID":"1b1d64eb-7388-44f0-8600-2dcbea6896c7","Type":"ContainerStarted","Data":"125bbe8ec24cd0e2628f02e2ff52129ee039666f3cd62c68a1b60bf2cea6f392"}
Jul 17 09:28:12.859071 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:12.858762    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/44a47c07-93d4-4047-9e80-3ffa7f6b066e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zvktb\" (UID: \"44a47c07-93d4-4047-9e80-3ffa7f6b066e\") " pod="openshift-insights/insights-runtime-extractor-zvktb"
Jul 17 09:28:12.861554 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:12.861524    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/44a47c07-93d4-4047-9e80-3ffa7f6b066e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zvktb\" (UID: \"44a47c07-93d4-4047-9e80-3ffa7f6b066e\") " pod="openshift-insights/insights-runtime-extractor-zvktb"
Jul 17 09:28:12.922550 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:12.922514    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gmxxn\""
Jul 17 09:28:12.929119 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:12.929096    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zvktb"
Jul 17 09:28:13.129820 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:13.129784    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zvktb"]
Jul 17 09:28:13.133353 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:28:13.133325    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44a47c07_93d4_4047_9e80_3ffa7f6b066e.slice/crio-adf62b55d7a949fe479b064de3d23cbdecc57fa3f85e1893096e5fbce1daae4e WatchSource:0}: Error finding container adf62b55d7a949fe479b064de3d23cbdecc57fa3f85e1893096e5fbce1daae4e: Status 404 returned error can't find the container with id adf62b55d7a949fe479b064de3d23cbdecc57fa3f85e1893096e5fbce1daae4e
Jul 17 09:28:13.232502 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:13.232469    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-689d766b54-g8vpn" event={"ID":"1b1d64eb-7388-44f0-8600-2dcbea6896c7","Type":"ContainerStarted","Data":"d9b3193cf6c9090b458e0b4fa8282c73d40adbc3df691b4d2b58312655805d7a"}
Jul 17 09:28:13.233870 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:13.233831    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zvktb" event={"ID":"44a47c07-93d4-4047-9e80-3ffa7f6b066e","Type":"ContainerStarted","Data":"adf62b55d7a949fe479b064de3d23cbdecc57fa3f85e1893096e5fbce1daae4e"}
Jul 17 09:28:13.250593 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:13.250547    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-689d766b54-g8vpn" podStartSLOduration=0.74729562 podStartE2EDuration="4.250534102s" totalImagesPullingTime="3.503238482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:28:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:13.249888462 +0000 UTC m=+90.832526839" watchObservedRunningTime="2026-07-17 09:28:13.250534102 +0000 UTC m=+90.833172478"
Jul 17 09:28:14.789980 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:14.789949    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-sm7n2"]
Jul 17 09:28:14.793741 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:14.793173    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-sm7n2"
Jul 17 09:28:14.797631 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:14.797611    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-4vp9x\""
Jul 17 09:28:14.797929 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:14.797912    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 17 09:28:14.801528 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:14.801509    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-sm7n2"]
Jul 17 09:28:14.876002 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:14.875972    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0cc618f8-0247-4eec-ae71-4a69b6495872-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-sm7n2\" (UID: \"0cc618f8-0247-4eec-ae71-4a69b6495872\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-sm7n2"
Jul 17 09:28:14.976969 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:14.976722    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0cc618f8-0247-4eec-ae71-4a69b6495872-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-sm7n2\" (UID: \"0cc618f8-0247-4eec-ae71-4a69b6495872\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-sm7n2"
Jul 17 09:28:14.976969 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:28:14.976874    2574 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 17 09:28:14.976969 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:28:14.976947    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cc618f8-0247-4eec-ae71-4a69b6495872-tls-certificates podName:0cc618f8-0247-4eec-ae71-4a69b6495872 nodeName:}" failed. No retries permitted until 2026-07-17 09:28:15.476926807 +0000 UTC m=+93.059565164 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/0cc618f8-0247-4eec-ae71-4a69b6495872-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-sm7n2" (UID: "0cc618f8-0247-4eec-ae71-4a69b6495872") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 17 09:28:15.481038 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:15.480607    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0cc618f8-0247-4eec-ae71-4a69b6495872-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-sm7n2\" (UID: \"0cc618f8-0247-4eec-ae71-4a69b6495872\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-sm7n2"
Jul 17 09:28:15.483901 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:15.483867    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/0cc618f8-0247-4eec-ae71-4a69b6495872-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-sm7n2\" (UID: \"0cc618f8-0247-4eec-ae71-4a69b6495872\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-sm7n2"
Jul 17 09:28:15.705669 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:15.705632    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-sm7n2"
Jul 17 09:28:19.054601 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.054566    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56f98c7866-ww6gm"]
Jul 17 09:28:19.057811 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.057782    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:28:19.067506 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.067358    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 17 09:28:19.068257 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.068206    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56f98c7866-ww6gm"]
Jul 17 09:28:19.109449 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.109427    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a83a14c-2f3b-4482-8557-131da3e63529-console-config\") pod \"console-56f98c7866-ww6gm\" (UID: \"5a83a14c-2f3b-4482-8557-131da3e63529\") " pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:28:19.109572 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.109476    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a83a14c-2f3b-4482-8557-131da3e63529-trusted-ca-bundle\") pod \"console-56f98c7866-ww6gm\" (UID: \"5a83a14c-2f3b-4482-8557-131da3e63529\") " pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:28:19.109630 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.109565    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a83a14c-2f3b-4482-8557-131da3e63529-console-serving-cert\") pod \"console-56f98c7866-ww6gm\" (UID: \"5a83a14c-2f3b-4482-8557-131da3e63529\") " pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:28:19.109630 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.109607    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a83a14c-2f3b-4482-8557-131da3e63529-console-oauth-config\") pod \"console-56f98c7866-ww6gm\" (UID: \"5a83a14c-2f3b-4482-8557-131da3e63529\") " pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:28:19.109718 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.109663    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a83a14c-2f3b-4482-8557-131da3e63529-oauth-serving-cert\") pod \"console-56f98c7866-ww6gm\" (UID: \"5a83a14c-2f3b-4482-8557-131da3e63529\") " pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:28:19.109718 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.109693    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a83a14c-2f3b-4482-8557-131da3e63529-service-ca\") pod \"console-56f98c7866-ww6gm\" (UID: \"5a83a14c-2f3b-4482-8557-131da3e63529\") " pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:28:19.109814 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.109721    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l8xc\" (UniqueName: \"kubernetes.io/projected/5a83a14c-2f3b-4482-8557-131da3e63529-kube-api-access-9l8xc\") pod \"console-56f98c7866-ww6gm\" (UID: \"5a83a14c-2f3b-4482-8557-131da3e63529\") " pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:28:19.210988 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.210952    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a83a14c-2f3b-4482-8557-131da3e63529-console-oauth-config\") pod \"console-56f98c7866-ww6gm\" (UID: \"5a83a14c-2f3b-4482-8557-131da3e63529\") " pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:28:19.211117 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.211009    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a83a14c-2f3b-4482-8557-131da3e63529-oauth-serving-cert\") pod \"console-56f98c7866-ww6gm\" (UID: \"5a83a14c-2f3b-4482-8557-131da3e63529\") " pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:28:19.211117 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.211039    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a83a14c-2f3b-4482-8557-131da3e63529-service-ca\") pod \"console-56f98c7866-ww6gm\" (UID: \"5a83a14c-2f3b-4482-8557-131da3e63529\") " pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:28:19.211117 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.211066    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9l8xc\" (UniqueName: \"kubernetes.io/projected/5a83a14c-2f3b-4482-8557-131da3e63529-kube-api-access-9l8xc\") pod \"console-56f98c7866-ww6gm\" (UID: \"5a83a14c-2f3b-4482-8557-131da3e63529\") " pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:28:19.211117 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.211095    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a83a14c-2f3b-4482-8557-131da3e63529-console-config\") pod \"console-56f98c7866-ww6gm\" (UID: \"5a83a14c-2f3b-4482-8557-131da3e63529\") " pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:28:19.211329 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.211278    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a83a14c-2f3b-4482-8557-131da3e63529-trusted-ca-bundle\") pod \"console-56f98c7866-ww6gm\" (UID: \"5a83a14c-2f3b-4482-8557-131da3e63529\") " pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:28:19.211370 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.211353    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a83a14c-2f3b-4482-8557-131da3e63529-console-serving-cert\") pod \"console-56f98c7866-ww6gm\" (UID: \"5a83a14c-2f3b-4482-8557-131da3e63529\") " pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:28:19.211946 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.211895    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a83a14c-2f3b-4482-8557-131da3e63529-service-ca\") pod \"console-56f98c7866-ww6gm\" (UID: \"5a83a14c-2f3b-4482-8557-131da3e63529\") " pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:28:19.212052 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.211980    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a83a14c-2f3b-4482-8557-131da3e63529-oauth-serving-cert\") pod \"console-56f98c7866-ww6gm\" (UID: \"5a83a14c-2f3b-4482-8557-131da3e63529\") " pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:28:19.212052 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.212023    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a83a14c-2f3b-4482-8557-131da3e63529-console-config\") pod \"console-56f98c7866-ww6gm\" (UID: \"5a83a14c-2f3b-4482-8557-131da3e63529\") " pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:28:19.212201 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.212171    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a83a14c-2f3b-4482-8557-131da3e63529-trusted-ca-bundle\") pod \"console-56f98c7866-ww6gm\" (UID: \"5a83a14c-2f3b-4482-8557-131da3e63529\") " pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:28:19.213654 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.213634    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a83a14c-2f3b-4482-8557-131da3e63529-console-oauth-config\") pod \"console-56f98c7866-ww6gm\" (UID: \"5a83a14c-2f3b-4482-8557-131da3e63529\") " pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:28:19.214345 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.214326    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a83a14c-2f3b-4482-8557-131da3e63529-console-serving-cert\") pod \"console-56f98c7866-ww6gm\" (UID: \"5a83a14c-2f3b-4482-8557-131da3e63529\") " pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:28:19.218978 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.218957    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l8xc\" (UniqueName: \"kubernetes.io/projected/5a83a14c-2f3b-4482-8557-131da3e63529-kube-api-access-9l8xc\") pod \"console-56f98c7866-ww6gm\" (UID: \"5a83a14c-2f3b-4482-8557-131da3e63529\") " pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:28:19.369760 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.369679    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:28:19.404444 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.404418    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-689d766b54-g8vpn"
Jul 17 09:28:19.404574 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.404471    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-689d766b54-g8vpn"
Jul 17 09:28:19.405998 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.405974    2574 patch_prober.go:28] interesting pod/console-689d766b54-g8vpn container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.14:8443/health\": dial tcp 10.132.0.14:8443: connect: connection refused" start-of-body=
Jul 17 09:28:19.406109 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:19.406023    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-689d766b54-g8vpn" podUID="1b1d64eb-7388-44f0-8600-2dcbea6896c7" containerName="console" probeResult="failure" output="Get \"https://10.132.0.14:8443/health\": dial tcp 10.132.0.14:8443: connect: connection refused"
Jul 17 09:28:22.742716 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:22.742667    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdac6d65-9077-4012-bcfb-b19556f35d5f-metrics-tls\") pod \"dns-default-gdlqf\" (UID: \"fdac6d65-9077-4012-bcfb-b19556f35d5f\") " pod="openshift-dns/dns-default-gdlqf"
Jul 17 09:28:22.743150 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:22.742755    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-registry-tls\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:28:22.745347 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:22.745319    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-registry-tls\") pod \"image-registry-75c884f5f7-7k78v\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") " pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:28:22.745476 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:22.745321    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fdac6d65-9077-4012-bcfb-b19556f35d5f-metrics-tls\") pod \"dns-default-gdlqf\" (UID: \"fdac6d65-9077-4012-bcfb-b19556f35d5f\") " pod="openshift-dns/dns-default-gdlqf"
Jul 17 09:28:23.026131 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:23.026062    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-45kr4\""
Jul 17 09:28:23.033575 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:23.033547    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:28:23.045756 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:23.045723    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-p8vfs\""
Jul 17 09:28:23.052389 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:23.052368    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gdlqf"
Jul 17 09:28:25.173576 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:25.173548    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qks8f"
Jul 17 09:28:25.550923 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:25.550898    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75c884f5f7-7k78v"]
Jul 17 09:28:25.553575 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:28:25.553528    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27c6d907_d449_4202_8700_dc369dadb9f4.slice/crio-603670d0b965930292f421166c387325105ce5a5cf8b012f621a76c21934ccb1 WatchSource:0}: Error finding container 603670d0b965930292f421166c387325105ce5a5cf8b012f621a76c21934ccb1: Status 404 returned error can't find the container with id 603670d0b965930292f421166c387325105ce5a5cf8b012f621a76c21934ccb1
Jul 17 09:28:25.774877 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:25.774763    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56f98c7866-ww6gm"]
Jul 17 09:28:25.778340 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:25.778275    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-sm7n2"]
Jul 17 09:28:25.778710 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:28:25.778669    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a83a14c_2f3b_4482_8557_131da3e63529.slice/crio-3eb06f794f6802f016ce4afee9ef85a7892e6b329cc67ad6c234d2e8d1f7a49c WatchSource:0}: Error finding container 3eb06f794f6802f016ce4afee9ef85a7892e6b329cc67ad6c234d2e8d1f7a49c: Status 404 returned error can't find the container with id 3eb06f794f6802f016ce4afee9ef85a7892e6b329cc67ad6c234d2e8d1f7a49c
Jul 17 09:28:25.779058 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:25.779033    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gdlqf"]
Jul 17 09:28:26.288924 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:26.288803    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zvktb" event={"ID":"44a47c07-93d4-4047-9e80-3ffa7f6b066e","Type":"ContainerStarted","Data":"468b608aa0e4723ee8e0b3e499d6b3404560d048448081769fe432a56939c5b7"}
Jul 17 09:28:26.295722 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:26.295645    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-sm7n2" event={"ID":"0cc618f8-0247-4eec-ae71-4a69b6495872","Type":"ContainerStarted","Data":"61cad92b19af8ed692b299db9b1337110fedf160d078e9491881fa10490a963a"}
Jul 17 09:28:26.299787 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:26.299651    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-hjfk5" event={"ID":"21d479b5-ee17-4fb5-acf9-541487651af1","Type":"ContainerStarted","Data":"fbd0342c5cea689072c271a2a5ed3d109feab9a625f757b5dd3ee7be45aa690c"}
Jul 17 09:28:26.300800 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:26.300532    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-hjfk5"
Jul 17 09:28:26.302991 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:26.302832    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gdlqf" event={"ID":"fdac6d65-9077-4012-bcfb-b19556f35d5f","Type":"ContainerStarted","Data":"f55f45f58eb379107f99c29336f294434176e2d19806f19b321a7c12be7e368f"}
Jul 17 09:28:26.305646 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:26.305153    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f98c7866-ww6gm" event={"ID":"5a83a14c-2f3b-4482-8557-131da3e63529","Type":"ContainerStarted","Data":"282fd27fd6e850c85709ef1c081219d0275384a8101ef1af2bc50b4fbf4e170c"}
Jul 17 09:28:26.305646 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:26.305192    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f98c7866-ww6gm" event={"ID":"5a83a14c-2f3b-4482-8557-131da3e63529","Type":"ContainerStarted","Data":"3eb06f794f6802f016ce4afee9ef85a7892e6b329cc67ad6c234d2e8d1f7a49c"}
Jul 17 09:28:26.307207 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:26.307141    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-hjfk5"
Jul 17 09:28:26.309072 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:26.308564    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75c884f5f7-7k78v" event={"ID":"27c6d907-d449-4202-8700-dc369dadb9f4","Type":"ContainerStarted","Data":"f2edd97a983faffb6e379bfce59d145c8f25a8fc224454f35f5c8358e2f43710"}
Jul 17 09:28:26.309072 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:26.308600    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75c884f5f7-7k78v" event={"ID":"27c6d907-d449-4202-8700-dc369dadb9f4","Type":"ContainerStarted","Data":"603670d0b965930292f421166c387325105ce5a5cf8b012f621a76c21934ccb1"}
Jul 17 09:28:26.309072 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:26.309028    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:28:26.322513 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:26.321981    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-hjfk5" podStartSLOduration=2.148171457 podStartE2EDuration="22.321963495s" totalImagesPullingTime="20.173792038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:28:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:26.31952458 +0000 UTC m=+103.902162958" watchObservedRunningTime="2026-07-17 09:28:26.321963495 +0000 UTC m=+103.904601872"
Jul 17 09:28:26.358694 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:26.358647    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-75c884f5f7-7k78v" podStartSLOduration=103.358633504 podStartE2EDuration="1m43.358633504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:26:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:26.340779285 +0000 UTC m=+103.923417661" watchObservedRunningTime="2026-07-17 09:28:26.358633504 +0000 UTC m=+103.941271876"
Jul 17 09:28:28.479776 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:28.479709    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56f98c7866-ww6gm" podStartSLOduration=9.47968727 podStartE2EDuration="9.47968727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:28:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:26.380256351 +0000 UTC m=+103.962894733" watchObservedRunningTime="2026-07-17 09:28:28.47968727 +0000 UTC m=+106.062325651"
Jul 17 09:28:28.487181 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:28.481570    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-75c884f5f7-7k78v"]
Jul 17 09:28:29.371426 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:29.370723    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:28:29.371426 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:29.370790    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:28:29.379179 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:29.379152    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:28:29.409608 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:29.409577    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-689d766b54-g8vpn"
Jul 17 09:28:29.414765 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:29.414712    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-689d766b54-g8vpn"
Jul 17 09:28:30.329908 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:30.329880    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:28:30.375808 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:30.375776    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-689d766b54-g8vpn"]
Jul 17 09:28:32.335158 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:32.335070    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gdlqf" event={"ID":"fdac6d65-9077-4012-bcfb-b19556f35d5f","Type":"ContainerStarted","Data":"6754fcd18b78ccf7a11b6f0c1f6fa15c9a0c32732c825a7ae1808d0c727a615a"}
Jul 17 09:28:32.335663 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:32.335186    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gdlqf" event={"ID":"fdac6d65-9077-4012-bcfb-b19556f35d5f","Type":"ContainerStarted","Data":"d3e63c2974de5db512cb6f41bdf064306ce96269250a3cbee47d9254a58269f8"}
Jul 17 09:28:32.335663 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:32.335220    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gdlqf"
Jul 17 09:28:32.337302 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:32.337278    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zvktb" event={"ID":"44a47c07-93d4-4047-9e80-3ffa7f6b066e","Type":"ContainerStarted","Data":"00fd77f8777c5e3953dc2224a579b26f30e5441083d4b2e4740622a554f77613"}
Jul 17 09:28:32.338996 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:32.338966    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-sm7n2" event={"ID":"0cc618f8-0247-4eec-ae71-4a69b6495872","Type":"ContainerStarted","Data":"5e7785bcfc9817a24c4ed382533dea69fff4b565f9f1633db0586bd2a1e01798"}
Jul 17 09:28:32.339289 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:32.339265    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-sm7n2"
Jul 17 09:28:32.344605 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:32.344580    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-sm7n2"
Jul 17 09:28:32.353292 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:32.353229    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-gdlqf" podStartSLOduration=68.556265194 podStartE2EDuration="1m14.35321281s" totalImagesPullingTime="5.796947616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:27:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:32.351983208 +0000 UTC m=+109.934621604" watchObservedRunningTime="2026-07-17 09:28:32.35321281 +0000 UTC m=+109.935851187"
Jul 17 09:28:32.366920 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:32.366882    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-sm7n2" podStartSLOduration=12.758083281 podStartE2EDuration="18.366869183s" totalImagesPullingTime="5.608785902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:28:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:32.365939037 +0000 UTC m=+109.948577440" watchObservedRunningTime="2026-07-17 09:28:32.366869183 +0000 UTC m=+109.949507560"
Jul 17 09:28:32.845231 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:32.845197    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-ctl9f"]
Jul 17 09:28:32.859259 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:32.859211    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-ctl9f"]
Jul 17 09:28:32.859662 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:32.859507    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-ctl9f"
Jul 17 09:28:32.862939 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:32.862298    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 17 09:28:32.862939 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:32.862560    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 17 09:28:32.862939 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:32.862791    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 17 09:28:32.863716 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:32.863671    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 17 09:28:32.864255 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:32.863845    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-jnpq2\""
Jul 17 09:28:32.864255 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:32.864089    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 17 09:28:33.035863 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:33.035821    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-ctl9f\" (UID: \"7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-ctl9f"
Jul 17 09:28:33.036099 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:33.035874    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-ctl9f\" (UID: \"7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-ctl9f"
Jul 17 09:28:33.036099 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:33.035910    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt2hb\" (UniqueName: \"kubernetes.io/projected/7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e-kube-api-access-dt2hb\") pod \"prometheus-operator-5945b5cfd-ctl9f\" (UID: \"7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-ctl9f"
Jul 17 09:28:33.036099 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:33.035968    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-ctl9f\" (UID: \"7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-ctl9f"
Jul 17 09:28:33.137015 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:33.136938    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-ctl9f\" (UID: \"7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-ctl9f"
Jul 17 09:28:33.137015 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:33.136995    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt2hb\" (UniqueName: \"kubernetes.io/projected/7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e-kube-api-access-dt2hb\") pod \"prometheus-operator-5945b5cfd-ctl9f\" (UID: \"7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-ctl9f"
Jul 17 09:28:33.137320 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:33.137049    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-ctl9f\" (UID: \"7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-ctl9f"
Jul 17 09:28:33.137320 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:33.137128    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-ctl9f\" (UID: \"7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-ctl9f"
Jul 17 09:28:33.137897 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:33.137871    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-ctl9f\" (UID: \"7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-ctl9f"
Jul 17 09:28:33.140322 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:33.140275    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-ctl9f\" (UID: \"7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-ctl9f"
Jul 17 09:28:33.147134 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:33.146983    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt2hb\" (UniqueName: \"kubernetes.io/projected/7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e-kube-api-access-dt2hb\") pod \"prometheus-operator-5945b5cfd-ctl9f\" (UID: \"7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-ctl9f"
Jul 17 09:28:33.151954 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:33.151931    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-ctl9f\" (UID: \"7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-ctl9f"
Jul 17 09:28:33.176882 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:33.176848    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-ctl9f"
Jul 17 09:28:33.355605 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:33.355572    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-ctl9f"]
Jul 17 09:28:33.359018 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:28:33.358986    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e4f20db_51a2_4b33_a7c1_bdbdbfd4eb0e.slice/crio-11125409a69f9947c02cf5c6896bd57b0764c22ff019c9c5b6660e7e43dc1ef0 WatchSource:0}: Error finding container 11125409a69f9947c02cf5c6896bd57b0764c22ff019c9c5b6660e7e43dc1ef0: Status 404 returned error can't find the container with id 11125409a69f9947c02cf5c6896bd57b0764c22ff019c9c5b6660e7e43dc1ef0
Jul 17 09:28:34.358007 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:34.357666    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-ctl9f" event={"ID":"7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e","Type":"ContainerStarted","Data":"11125409a69f9947c02cf5c6896bd57b0764c22ff019c9c5b6660e7e43dc1ef0"}
Jul 17 09:28:37.368389 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:37.368356    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-ctl9f" event={"ID":"7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e","Type":"ContainerStarted","Data":"9972b564ff0b9b999643c6ef9ed34066e77a6704d4e7d3a8476de7d50d39ead8"}
Jul 17 09:28:37.370626 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:37.370603    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zvktb" event={"ID":"44a47c07-93d4-4047-9e80-3ffa7f6b066e","Type":"ContainerStarted","Data":"13f5312af1700907f37e756f85f88eb5c2871b78b6c732b78815eb4673a0415e"}
Jul 17 09:28:37.389074 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:37.389037    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zvktb" podStartSLOduration=32.588299595 podStartE2EDuration="56.38902463s" totalImagesPullingTime="23.800725035s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 09:27:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:37.387956195 +0000 UTC m=+114.970594588" watchObservedRunningTime="2026-07-17 09:28:37.38902463 +0000 UTC m=+114.971663006"
Jul 17 09:28:38.376086 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:38.376052    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-ctl9f" event={"ID":"7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e","Type":"ContainerStarted","Data":"643c4a8e00522f3457eca832f0fd040a9374bed4ea42a59b196106f07cdde49f"}
Jul 17 09:28:38.393155 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:38.393110    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-ctl9f" podStartSLOduration=2.560231909 podStartE2EDuration="6.393095743s" totalImagesPullingTime="3.832863834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:28:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:38.392470602 +0000 UTC m=+115.975108978" watchObservedRunningTime="2026-07-17 09:28:38.393095743 +0000 UTC m=+115.975734119"
Jul 17 09:28:40.174934 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.174892    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-b5n86"]
Jul 17 09:28:40.198479 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.198454    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-b5n86"]
Jul 17 09:28:40.198633 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.198587    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-b5n86"
Jul 17 09:28:40.201153 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.201135    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 17 09:28:40.202401 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.202381    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 17 09:28:40.202467 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.202453    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-xxnwn\""
Jul 17 09:28:40.212151 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.212133    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-bmcbf"]
Jul 17 09:28:40.234764 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.234744    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.237521 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.237483    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 17 09:28:40.237633 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.237615    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 17 09:28:40.237684 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.237663    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 17 09:28:40.237726 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.237702    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pf7jj\""
Jul 17 09:28:40.299552 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.299528    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c644d32e-0c8d-403f-ae35-5d7cff59d490-node-exporter-accelerators-collector-config\") pod \"node-exporter-bmcbf\" (UID: \"c644d32e-0c8d-403f-ae35-5d7cff59d490\") " pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.299642 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.299561    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/200c4e96-7d88-49ff-8990-8bd65d3c2817-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-b5n86\" (UID: \"200c4e96-7d88-49ff-8990-8bd65d3c2817\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-b5n86"
Jul 17 09:28:40.299642 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.299581    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c644d32e-0c8d-403f-ae35-5d7cff59d490-metrics-client-ca\") pod \"node-exporter-bmcbf\" (UID: \"c644d32e-0c8d-403f-ae35-5d7cff59d490\") " pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.299642 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.299608    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c644d32e-0c8d-403f-ae35-5d7cff59d490-node-exporter-textfile\") pod \"node-exporter-bmcbf\" (UID: \"c644d32e-0c8d-403f-ae35-5d7cff59d490\") " pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.299642 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.299637    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c644d32e-0c8d-403f-ae35-5d7cff59d490-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bmcbf\" (UID: \"c644d32e-0c8d-403f-ae35-5d7cff59d490\") " pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.299813 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.299662    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4k2t\" (UniqueName: \"kubernetes.io/projected/c644d32e-0c8d-403f-ae35-5d7cff59d490-kube-api-access-j4k2t\") pod \"node-exporter-bmcbf\" (UID: \"c644d32e-0c8d-403f-ae35-5d7cff59d490\") " pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.299813 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.299718    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c644d32e-0c8d-403f-ae35-5d7cff59d490-sys\") pod \"node-exporter-bmcbf\" (UID: \"c644d32e-0c8d-403f-ae35-5d7cff59d490\") " pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.299813 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.299733    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c644d32e-0c8d-403f-ae35-5d7cff59d490-node-exporter-tls\") pod \"node-exporter-bmcbf\" (UID: \"c644d32e-0c8d-403f-ae35-5d7cff59d490\") " pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.299813 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.299750    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/200c4e96-7d88-49ff-8990-8bd65d3c2817-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-b5n86\" (UID: \"200c4e96-7d88-49ff-8990-8bd65d3c2817\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-b5n86"
Jul 17 09:28:40.299813 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.299801    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/200c4e96-7d88-49ff-8990-8bd65d3c2817-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-b5n86\" (UID: \"200c4e96-7d88-49ff-8990-8bd65d3c2817\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-b5n86"
Jul 17 09:28:40.300031 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.299853    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c644d32e-0c8d-403f-ae35-5d7cff59d490-root\") pod \"node-exporter-bmcbf\" (UID: \"c644d32e-0c8d-403f-ae35-5d7cff59d490\") " pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.300031 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.299877    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c644d32e-0c8d-403f-ae35-5d7cff59d490-node-exporter-wtmp\") pod \"node-exporter-bmcbf\" (UID: \"c644d32e-0c8d-403f-ae35-5d7cff59d490\") " pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.300031 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.299913    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvzhd\" (UniqueName: \"kubernetes.io/projected/200c4e96-7d88-49ff-8990-8bd65d3c2817-kube-api-access-zvzhd\") pod \"openshift-state-metrics-79f6958cdf-b5n86\" (UID: \"200c4e96-7d88-49ff-8990-8bd65d3c2817\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-b5n86"
Jul 17 09:28:40.400488 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.400466    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c644d32e-0c8d-403f-ae35-5d7cff59d490-node-exporter-accelerators-collector-config\") pod \"node-exporter-bmcbf\" (UID: \"c644d32e-0c8d-403f-ae35-5d7cff59d490\") " pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.400592 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.400495    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/200c4e96-7d88-49ff-8990-8bd65d3c2817-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-b5n86\" (UID: \"200c4e96-7d88-49ff-8990-8bd65d3c2817\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-b5n86"
Jul 17 09:28:40.400592 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.400512    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c644d32e-0c8d-403f-ae35-5d7cff59d490-metrics-client-ca\") pod \"node-exporter-bmcbf\" (UID: \"c644d32e-0c8d-403f-ae35-5d7cff59d490\") " pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.400592 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.400529    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c644d32e-0c8d-403f-ae35-5d7cff59d490-node-exporter-textfile\") pod \"node-exporter-bmcbf\" (UID: \"c644d32e-0c8d-403f-ae35-5d7cff59d490\") " pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.400592 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.400577    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c644d32e-0c8d-403f-ae35-5d7cff59d490-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bmcbf\" (UID: \"c644d32e-0c8d-403f-ae35-5d7cff59d490\") " pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.400802 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.400608    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4k2t\" (UniqueName: \"kubernetes.io/projected/c644d32e-0c8d-403f-ae35-5d7cff59d490-kube-api-access-j4k2t\") pod \"node-exporter-bmcbf\" (UID: \"c644d32e-0c8d-403f-ae35-5d7cff59d490\") " pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.400802 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.400637    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c644d32e-0c8d-403f-ae35-5d7cff59d490-sys\") pod \"node-exporter-bmcbf\" (UID: \"c644d32e-0c8d-403f-ae35-5d7cff59d490\") " pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.400802 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.400655    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c644d32e-0c8d-403f-ae35-5d7cff59d490-node-exporter-tls\") pod \"node-exporter-bmcbf\" (UID: \"c644d32e-0c8d-403f-ae35-5d7cff59d490\") " pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.400802 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.400685    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/200c4e96-7d88-49ff-8990-8bd65d3c2817-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-b5n86\" (UID: \"200c4e96-7d88-49ff-8990-8bd65d3c2817\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-b5n86"
Jul 17 09:28:40.400802 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.400712    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/200c4e96-7d88-49ff-8990-8bd65d3c2817-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-b5n86\" (UID: \"200c4e96-7d88-49ff-8990-8bd65d3c2817\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-b5n86"
Jul 17 09:28:40.400802 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.400722    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c644d32e-0c8d-403f-ae35-5d7cff59d490-sys\") pod \"node-exporter-bmcbf\" (UID: \"c644d32e-0c8d-403f-ae35-5d7cff59d490\") " pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.400802 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.400758    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c644d32e-0c8d-403f-ae35-5d7cff59d490-root\") pod \"node-exporter-bmcbf\" (UID: \"c644d32e-0c8d-403f-ae35-5d7cff59d490\") " pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.400802 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.400782    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c644d32e-0c8d-403f-ae35-5d7cff59d490-node-exporter-wtmp\") pod \"node-exporter-bmcbf\" (UID: \"c644d32e-0c8d-403f-ae35-5d7cff59d490\") " pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.401266 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.400817    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvzhd\" (UniqueName: \"kubernetes.io/projected/200c4e96-7d88-49ff-8990-8bd65d3c2817-kube-api-access-zvzhd\") pod \"openshift-state-metrics-79f6958cdf-b5n86\" (UID: \"200c4e96-7d88-49ff-8990-8bd65d3c2817\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-b5n86"
Jul 17 09:28:40.401266 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:28:40.400848    2574 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 17 09:28:40.401266 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:28:40.400925    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/200c4e96-7d88-49ff-8990-8bd65d3c2817-openshift-state-metrics-tls podName:200c4e96-7d88-49ff-8990-8bd65d3c2817 nodeName:}" failed. No retries permitted until 2026-07-17 09:28:40.900886784 +0000 UTC m=+118.483525138 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/200c4e96-7d88-49ff-8990-8bd65d3c2817-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-b5n86" (UID: "200c4e96-7d88-49ff-8990-8bd65d3c2817") : secret "openshift-state-metrics-tls" not found
Jul 17 09:28:40.401266 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:28:40.400930    2574 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 17 09:28:40.401266 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:28:40.400990    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c644d32e-0c8d-403f-ae35-5d7cff59d490-node-exporter-tls podName:c644d32e-0c8d-403f-ae35-5d7cff59d490 nodeName:}" failed. No retries permitted until 2026-07-17 09:28:40.900974626 +0000 UTC m=+118.483612981 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/c644d32e-0c8d-403f-ae35-5d7cff59d490-node-exporter-tls") pod "node-exporter-bmcbf" (UID: "c644d32e-0c8d-403f-ae35-5d7cff59d490") : secret "node-exporter-tls" not found
Jul 17 09:28:40.401266 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.401038    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c644d32e-0c8d-403f-ae35-5d7cff59d490-root\") pod \"node-exporter-bmcbf\" (UID: \"c644d32e-0c8d-403f-ae35-5d7cff59d490\") " pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.401266 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.401093    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c644d32e-0c8d-403f-ae35-5d7cff59d490-metrics-client-ca\") pod \"node-exporter-bmcbf\" (UID: \"c644d32e-0c8d-403f-ae35-5d7cff59d490\") " pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.401266 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.401150    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c644d32e-0c8d-403f-ae35-5d7cff59d490-node-exporter-accelerators-collector-config\") pod \"node-exporter-bmcbf\" (UID: \"c644d32e-0c8d-403f-ae35-5d7cff59d490\") " pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.401266 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.401182    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c644d32e-0c8d-403f-ae35-5d7cff59d490-node-exporter-wtmp\") pod \"node-exporter-bmcbf\" (UID: \"c644d32e-0c8d-403f-ae35-5d7cff59d490\") " pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.401266 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.401265    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/200c4e96-7d88-49ff-8990-8bd65d3c2817-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-b5n86\" (UID: \"200c4e96-7d88-49ff-8990-8bd65d3c2817\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-b5n86"
Jul 17 09:28:40.401602 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.401346    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c644d32e-0c8d-403f-ae35-5d7cff59d490-node-exporter-textfile\") pod \"node-exporter-bmcbf\" (UID: \"c644d32e-0c8d-403f-ae35-5d7cff59d490\") " pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.402954 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.402937    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c644d32e-0c8d-403f-ae35-5d7cff59d490-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bmcbf\" (UID: \"c644d32e-0c8d-403f-ae35-5d7cff59d490\") " pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.403213 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.403193    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/200c4e96-7d88-49ff-8990-8bd65d3c2817-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-b5n86\" (UID: \"200c4e96-7d88-49ff-8990-8bd65d3c2817\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-b5n86"
Jul 17 09:28:40.409550 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.409531    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4k2t\" (UniqueName: \"kubernetes.io/projected/c644d32e-0c8d-403f-ae35-5d7cff59d490-kube-api-access-j4k2t\") pod \"node-exporter-bmcbf\" (UID: \"c644d32e-0c8d-403f-ae35-5d7cff59d490\") " pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.410104 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.410087    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvzhd\" (UniqueName: \"kubernetes.io/projected/200c4e96-7d88-49ff-8990-8bd65d3c2817-kube-api-access-zvzhd\") pod \"openshift-state-metrics-79f6958cdf-b5n86\" (UID: \"200c4e96-7d88-49ff-8990-8bd65d3c2817\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-b5n86"
Jul 17 09:28:40.903549 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.903519    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c644d32e-0c8d-403f-ae35-5d7cff59d490-node-exporter-tls\") pod \"node-exporter-bmcbf\" (UID: \"c644d32e-0c8d-403f-ae35-5d7cff59d490\") " pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.903677 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.903556    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/200c4e96-7d88-49ff-8990-8bd65d3c2817-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-b5n86\" (UID: \"200c4e96-7d88-49ff-8990-8bd65d3c2817\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-b5n86"
Jul 17 09:28:40.905904 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.905877    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c644d32e-0c8d-403f-ae35-5d7cff59d490-node-exporter-tls\") pod \"node-exporter-bmcbf\" (UID: \"c644d32e-0c8d-403f-ae35-5d7cff59d490\") " pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:40.906036 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:40.906018    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/200c4e96-7d88-49ff-8990-8bd65d3c2817-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-b5n86\" (UID: \"200c4e96-7d88-49ff-8990-8bd65d3c2817\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-b5n86"
Jul 17 09:28:41.107422 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:41.107399    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-b5n86"
Jul 17 09:28:41.143593 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:41.143572    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bmcbf"
Jul 17 09:28:41.171428 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:28:41.171406    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc644d32e_0c8d_403f_ae35_5d7cff59d490.slice/crio-f4519e118773d42630782231f7ff5d70fa7a14f75b377c893a0cd0e428cbcd25 WatchSource:0}: Error finding container f4519e118773d42630782231f7ff5d70fa7a14f75b377c893a0cd0e428cbcd25: Status 404 returned error can't find the container with id f4519e118773d42630782231f7ff5d70fa7a14f75b377c893a0cd0e428cbcd25
Jul 17 09:28:41.271596 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:41.271412    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-b5n86"]
Jul 17 09:28:41.274201 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:28:41.274175    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod200c4e96_7d88_49ff_8990_8bd65d3c2817.slice/crio-c23e4e693bcf5b1bcf8f2a9fabfe727e05673691c9614767f04762ac86cac5c2 WatchSource:0}: Error finding container c23e4e693bcf5b1bcf8f2a9fabfe727e05673691c9614767f04762ac86cac5c2: Status 404 returned error can't find the container with id c23e4e693bcf5b1bcf8f2a9fabfe727e05673691c9614767f04762ac86cac5c2
Jul 17 09:28:41.386495 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:41.386459    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bmcbf" event={"ID":"c644d32e-0c8d-403f-ae35-5d7cff59d490","Type":"ContainerStarted","Data":"f4519e118773d42630782231f7ff5d70fa7a14f75b377c893a0cd0e428cbcd25"}
Jul 17 09:28:41.388179 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:41.388152    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-b5n86" event={"ID":"200c4e96-7d88-49ff-8990-8bd65d3c2817","Type":"ContainerStarted","Data":"c23e4e693bcf5b1bcf8f2a9fabfe727e05673691c9614767f04762ac86cac5c2"}
Jul 17 09:28:42.346750 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:42.346718    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gdlqf"
Jul 17 09:28:42.394021 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:42.393980    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-b5n86" event={"ID":"200c4e96-7d88-49ff-8990-8bd65d3c2817","Type":"ContainerStarted","Data":"15c95f3886664df2477c8cbcb7ec824d2d068fc04d27bc43aef75f2cd8efcdfc"}
Jul 17 09:28:42.394195 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:42.394048    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-b5n86" event={"ID":"200c4e96-7d88-49ff-8990-8bd65d3c2817","Type":"ContainerStarted","Data":"728bf689d821b453c54b4e8b299f6252bad35b0c0ca2268057f2da5451463897"}
Jul 17 09:28:43.398569 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:43.398533    2574 generic.go:356] "Generic (PLEG): container finished" podID="c644d32e-0c8d-403f-ae35-5d7cff59d490" containerID="98df6968bbbf07a9658827761de4c5b8a16afd6b3ede751fc3271b176efb06ab" exitCode=0
Jul 17 09:28:43.398946 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:43.398623    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bmcbf" event={"ID":"c644d32e-0c8d-403f-ae35-5d7cff59d490","Type":"ContainerDied","Data":"98df6968bbbf07a9658827761de4c5b8a16afd6b3ede751fc3271b176efb06ab"}
Jul 17 09:28:44.404319 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.404282    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bmcbf" event={"ID":"c644d32e-0c8d-403f-ae35-5d7cff59d490","Type":"ContainerStarted","Data":"68669c53a66b1afb92e0427649a7a82ad1b3e2bc1bc8d0deffea361304176998"}
Jul 17 09:28:44.404726 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.404338    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bmcbf" event={"ID":"c644d32e-0c8d-403f-ae35-5d7cff59d490","Type":"ContainerStarted","Data":"5303c0b444545d0997bcc17242296e352721e4f42b0fa14e799c317c00538e4e"}
Jul 17 09:28:44.406270 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.406220    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-b5n86" event={"ID":"200c4e96-7d88-49ff-8990-8bd65d3c2817","Type":"ContainerStarted","Data":"a23edd72bd6b929988555b36a9e54adb8bc0efb44c6b9981f0d4934ca884ead2"}
Jul 17 09:28:44.422179 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.422120    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-bmcbf" podStartSLOduration=2.947085625 podStartE2EDuration="4.422107146s" totalImagesPullingTime="1.350832222s" totalInitContainerRuntime="124.189299ms" isStatefulPod=true podCreationTimestamp="2026-07-17 09:28:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:44.421068222 +0000 UTC m=+122.003706598" watchObservedRunningTime="2026-07-17 09:28:44.422107146 +0000 UTC m=+122.004745559"
Jul 17 09:28:44.441256 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.438496    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-b5n86" podStartSLOduration=2.257793402 podStartE2EDuration="4.438478403s" totalImagesPullingTime="2.180685001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:28:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:44.437157523 +0000 UTC m=+122.019795899" watchObservedRunningTime="2026-07-17 09:28:44.438478403 +0000 UTC m=+122.021116781"
Jul 17 09:28:44.588491 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.588461    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-64874587b8-krgcd"]
Jul 17 09:28:44.610039 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.610021    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-64874587b8-krgcd"]
Jul 17 09:28:44.610130 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.610094    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-64874587b8-krgcd"
Jul 17 09:28:44.612868 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.612846    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 17 09:28:44.612986 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.612901    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 17 09:28:44.612986 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.612938    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-skspr\""
Jul 17 09:28:44.613167 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.613144    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 17 09:28:44.613261 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.613207    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ctjdi3f7rh1vf\""
Jul 17 09:28:44.613413 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.613398    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 17 09:28:44.630664 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.630643    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-64874587b8-krgcd\" (UID: \"d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b\") " pod="openshift-monitoring/metrics-server-64874587b8-krgcd"
Jul 17 09:28:44.630761 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.630677    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b-secret-metrics-server-client-certs\") pod \"metrics-server-64874587b8-krgcd\" (UID: \"d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b\") " pod="openshift-monitoring/metrics-server-64874587b8-krgcd"
Jul 17 09:28:44.630761 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.630702    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b-audit-log\") pod \"metrics-server-64874587b8-krgcd\" (UID: \"d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b\") " pod="openshift-monitoring/metrics-server-64874587b8-krgcd"
Jul 17 09:28:44.630761 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.630729    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b-metrics-server-audit-profiles\") pod \"metrics-server-64874587b8-krgcd\" (UID: \"d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b\") " pod="openshift-monitoring/metrics-server-64874587b8-krgcd"
Jul 17 09:28:44.630907 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.630763    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b-client-ca-bundle\") pod \"metrics-server-64874587b8-krgcd\" (UID: \"d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b\") " pod="openshift-monitoring/metrics-server-64874587b8-krgcd"
Jul 17 09:28:44.630907 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.630801    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b-secret-metrics-server-tls\") pod \"metrics-server-64874587b8-krgcd\" (UID: \"d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b\") " pod="openshift-monitoring/metrics-server-64874587b8-krgcd"
Jul 17 09:28:44.630907 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.630829    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b4vc\" (UniqueName: \"kubernetes.io/projected/d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b-kube-api-access-8b4vc\") pod \"metrics-server-64874587b8-krgcd\" (UID: \"d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b\") " pod="openshift-monitoring/metrics-server-64874587b8-krgcd"
Jul 17 09:28:44.731606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.731587    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-64874587b8-krgcd\" (UID: \"d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b\") " pod="openshift-monitoring/metrics-server-64874587b8-krgcd"
Jul 17 09:28:44.731713 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.731615    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b-secret-metrics-server-client-certs\") pod \"metrics-server-64874587b8-krgcd\" (UID: \"d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b\") " pod="openshift-monitoring/metrics-server-64874587b8-krgcd"
Jul 17 09:28:44.731713 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.731631    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b-audit-log\") pod \"metrics-server-64874587b8-krgcd\" (UID: \"d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b\") " pod="openshift-monitoring/metrics-server-64874587b8-krgcd"
Jul 17 09:28:44.731713 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.731660    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b-metrics-server-audit-profiles\") pod \"metrics-server-64874587b8-krgcd\" (UID: \"d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b\") " pod="openshift-monitoring/metrics-server-64874587b8-krgcd"
Jul 17 09:28:44.731713 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.731693    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b-client-ca-bundle\") pod \"metrics-server-64874587b8-krgcd\" (UID: \"d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b\") " pod="openshift-monitoring/metrics-server-64874587b8-krgcd"
Jul 17 09:28:44.731899 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.731727    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b-secret-metrics-server-tls\") pod \"metrics-server-64874587b8-krgcd\" (UID: \"d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b\") " pod="openshift-monitoring/metrics-server-64874587b8-krgcd"
Jul 17 09:28:44.731899 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.731749    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b4vc\" (UniqueName: \"kubernetes.io/projected/d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b-kube-api-access-8b4vc\") pod \"metrics-server-64874587b8-krgcd\" (UID: \"d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b\") " pod="openshift-monitoring/metrics-server-64874587b8-krgcd"
Jul 17 09:28:44.732106 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.732083    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b-audit-log\") pod \"metrics-server-64874587b8-krgcd\" (UID: \"d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b\") " pod="openshift-monitoring/metrics-server-64874587b8-krgcd"
Jul 17 09:28:44.732581 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.732556    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-64874587b8-krgcd\" (UID: \"d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b\") " pod="openshift-monitoring/metrics-server-64874587b8-krgcd"
Jul 17 09:28:44.732701 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.732683    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b-metrics-server-audit-profiles\") pod \"metrics-server-64874587b8-krgcd\" (UID: \"d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b\") " pod="openshift-monitoring/metrics-server-64874587b8-krgcd"
Jul 17 09:28:44.734146 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.734126    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b-secret-metrics-server-tls\") pod \"metrics-server-64874587b8-krgcd\" (UID: \"d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b\") " pod="openshift-monitoring/metrics-server-64874587b8-krgcd"
Jul 17 09:28:44.734448 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.734427    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b-client-ca-bundle\") pod \"metrics-server-64874587b8-krgcd\" (UID: \"d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b\") " pod="openshift-monitoring/metrics-server-64874587b8-krgcd"
Jul 17 09:28:44.734494 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.734452    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b-secret-metrics-server-client-certs\") pod \"metrics-server-64874587b8-krgcd\" (UID: \"d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b\") " pod="openshift-monitoring/metrics-server-64874587b8-krgcd"
Jul 17 09:28:44.739069 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.739049    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b4vc\" (UniqueName: \"kubernetes.io/projected/d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b-kube-api-access-8b4vc\") pod \"metrics-server-64874587b8-krgcd\" (UID: \"d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b\") " pod="openshift-monitoring/metrics-server-64874587b8-krgcd"
Jul 17 09:28:44.920376 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:44.920354    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-64874587b8-krgcd"
Jul 17 09:28:45.084031 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.084005    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-64874587b8-krgcd"]
Jul 17 09:28:45.085898 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:28:45.085870    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8bfa5bb_c0f1_41be_8faf_c6bd5206f92b.slice/crio-a32b528ad5a6e86763eb93edab38d404f597844d867f19757ae25e52f5162342 WatchSource:0}: Error finding container a32b528ad5a6e86763eb93edab38d404f597844d867f19757ae25e52f5162342: Status 404 returned error can't find the container with id a32b528ad5a6e86763eb93edab38d404f597844d867f19757ae25e52f5162342
Jul 17 09:28:45.390286 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.390207    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6966f9675-gcrpl"]
Jul 17 09:28:45.417868 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.417833    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-64874587b8-krgcd" event={"ID":"d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b","Type":"ContainerStarted","Data":"a32b528ad5a6e86763eb93edab38d404f597844d867f19757ae25e52f5162342"}
Jul 17 09:28:45.418277 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.417876    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6966f9675-gcrpl"]
Jul 17 09:28:45.418277 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.417965    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl"
Jul 17 09:28:45.420731 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.420715    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 17 09:28:45.420815 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.420715    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 17 09:28:45.420881 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.420828    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 17 09:28:45.420934 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.420878    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 17 09:28:45.421104 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.421079    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 17 09:28:45.421207 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.421119    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-jn46p\""
Jul 17 09:28:45.425617 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.425589    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 17 09:28:45.437142 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.437119    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c6c7a08a-8f34-41b0-99cd-9613bf032f67-telemeter-client-tls\") pod \"telemeter-client-6966f9675-gcrpl\" (UID: \"c6c7a08a-8f34-41b0-99cd-9613bf032f67\") " pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl"
Jul 17 09:28:45.437262 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.437177    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gsll\" (UniqueName: \"kubernetes.io/projected/c6c7a08a-8f34-41b0-99cd-9613bf032f67-kube-api-access-2gsll\") pod \"telemeter-client-6966f9675-gcrpl\" (UID: \"c6c7a08a-8f34-41b0-99cd-9613bf032f67\") " pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl"
Jul 17 09:28:45.437334 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.437307    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c6c7a08a-8f34-41b0-99cd-9613bf032f67-federate-client-tls\") pod \"telemeter-client-6966f9675-gcrpl\" (UID: \"c6c7a08a-8f34-41b0-99cd-9613bf032f67\") " pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl"
Jul 17 09:28:45.437391 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.437378    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c6c7a08a-8f34-41b0-99cd-9613bf032f67-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6966f9675-gcrpl\" (UID: \"c6c7a08a-8f34-41b0-99cd-9613bf032f67\") " pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl"
Jul 17 09:28:45.437443 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.437416    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6c7a08a-8f34-41b0-99cd-9613bf032f67-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6966f9675-gcrpl\" (UID: \"c6c7a08a-8f34-41b0-99cd-9613bf032f67\") " pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl"
Jul 17 09:28:45.437443 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.437436    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6c7a08a-8f34-41b0-99cd-9613bf032f67-metrics-client-ca\") pod \"telemeter-client-6966f9675-gcrpl\" (UID: \"c6c7a08a-8f34-41b0-99cd-9613bf032f67\") " pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl"
Jul 17 09:28:45.437569 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.437545    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6c7a08a-8f34-41b0-99cd-9613bf032f67-serving-certs-ca-bundle\") pod \"telemeter-client-6966f9675-gcrpl\" (UID: \"c6c7a08a-8f34-41b0-99cd-9613bf032f67\") " pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl"
Jul 17 09:28:45.438131 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.437715    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c6c7a08a-8f34-41b0-99cd-9613bf032f67-secret-telemeter-client\") pod \"telemeter-client-6966f9675-gcrpl\" (UID: \"c6c7a08a-8f34-41b0-99cd-9613bf032f67\") " pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl"
Jul 17 09:28:45.538662 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.538636    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c6c7a08a-8f34-41b0-99cd-9613bf032f67-telemeter-client-tls\") pod \"telemeter-client-6966f9675-gcrpl\" (UID: \"c6c7a08a-8f34-41b0-99cd-9613bf032f67\") " pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl"
Jul 17 09:28:45.538742 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.538672    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gsll\" (UniqueName: \"kubernetes.io/projected/c6c7a08a-8f34-41b0-99cd-9613bf032f67-kube-api-access-2gsll\") pod \"telemeter-client-6966f9675-gcrpl\" (UID: \"c6c7a08a-8f34-41b0-99cd-9613bf032f67\") " pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl"
Jul 17 09:28:45.538742 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.538704    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c6c7a08a-8f34-41b0-99cd-9613bf032f67-federate-client-tls\") pod \"telemeter-client-6966f9675-gcrpl\" (UID: \"c6c7a08a-8f34-41b0-99cd-9613bf032f67\") " pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl"
Jul 17 09:28:45.538742 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.538733    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c6c7a08a-8f34-41b0-99cd-9613bf032f67-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6966f9675-gcrpl\" (UID: \"c6c7a08a-8f34-41b0-99cd-9613bf032f67\") " pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl"
Jul 17 09:28:45.538868 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.538751    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6c7a08a-8f34-41b0-99cd-9613bf032f67-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6966f9675-gcrpl\" (UID: \"c6c7a08a-8f34-41b0-99cd-9613bf032f67\") " pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl"
Jul 17 09:28:45.538868 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.538768    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6c7a08a-8f34-41b0-99cd-9613bf032f67-metrics-client-ca\") pod \"telemeter-client-6966f9675-gcrpl\" (UID: \"c6c7a08a-8f34-41b0-99cd-9613bf032f67\") " pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl"
Jul 17 09:28:45.538868 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.538794    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6c7a08a-8f34-41b0-99cd-9613bf032f67-serving-certs-ca-bundle\") pod \"telemeter-client-6966f9675-gcrpl\" (UID: \"c6c7a08a-8f34-41b0-99cd-9613bf032f67\") " pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl"
Jul 17 09:28:45.538868 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.538851    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c6c7a08a-8f34-41b0-99cd-9613bf032f67-secret-telemeter-client\") pod \"telemeter-client-6966f9675-gcrpl\" (UID: \"c6c7a08a-8f34-41b0-99cd-9613bf032f67\") " pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl"
Jul 17 09:28:45.539779 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.539755    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c6c7a08a-8f34-41b0-99cd-9613bf032f67-metrics-client-ca\") pod \"telemeter-client-6966f9675-gcrpl\" (UID: \"c6c7a08a-8f34-41b0-99cd-9613bf032f67\") " pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl"
Jul 17 09:28:45.539887 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.539827    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6c7a08a-8f34-41b0-99cd-9613bf032f67-serving-certs-ca-bundle\") pod \"telemeter-client-6966f9675-gcrpl\" (UID: \"c6c7a08a-8f34-41b0-99cd-9613bf032f67\") " pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl"
Jul 17 09:28:45.540124 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.540096    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c6c7a08a-8f34-41b0-99cd-9613bf032f67-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6966f9675-gcrpl\" (UID: \"c6c7a08a-8f34-41b0-99cd-9613bf032f67\") " pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl"
Jul 17 09:28:45.541299 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.541276    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/c6c7a08a-8f34-41b0-99cd-9613bf032f67-telemeter-client-tls\") pod \"telemeter-client-6966f9675-gcrpl\" (UID: \"c6c7a08a-8f34-41b0-99cd-9613bf032f67\") " pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl"
Jul 17 09:28:45.541299 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.541295    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c6c7a08a-8f34-41b0-99cd-9613bf032f67-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6966f9675-gcrpl\" (UID: \"c6c7a08a-8f34-41b0-99cd-9613bf032f67\") " pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl"
Jul 17 09:28:45.541422 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.541403    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/c6c7a08a-8f34-41b0-99cd-9613bf032f67-secret-telemeter-client\") pod \"telemeter-client-6966f9675-gcrpl\" (UID: \"c6c7a08a-8f34-41b0-99cd-9613bf032f67\") " pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl"
Jul 17 09:28:45.541483 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.541468    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/c6c7a08a-8f34-41b0-99cd-9613bf032f67-federate-client-tls\") pod \"telemeter-client-6966f9675-gcrpl\" (UID: \"c6c7a08a-8f34-41b0-99cd-9613bf032f67\") " pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl"
Jul 17 09:28:45.546404 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.546387    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gsll\" (UniqueName: \"kubernetes.io/projected/c6c7a08a-8f34-41b0-99cd-9613bf032f67-kube-api-access-2gsll\") pod \"telemeter-client-6966f9675-gcrpl\" (UID: \"c6c7a08a-8f34-41b0-99cd-9613bf032f67\") " pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl"
Jul 17 09:28:45.727326 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.727305    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl"
Jul 17 09:28:45.889829 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:45.889798    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6966f9675-gcrpl"]
Jul 17 09:28:45.891253 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:28:45.891209    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6c7a08a_8f34_41b0_99cd_9613bf032f67.slice/crio-bcb71c2de89f3d8dc0a898dd6f4235b6a48cc3778e0cb9bbe9d4e53c13e28aa6 WatchSource:0}: Error finding container bcb71c2de89f3d8dc0a898dd6f4235b6a48cc3778e0cb9bbe9d4e53c13e28aa6: Status 404 returned error can't find the container with id bcb71c2de89f3d8dc0a898dd6f4235b6a48cc3778e0cb9bbe9d4e53c13e28aa6
Jul 17 09:28:46.420507 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.420425    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl" event={"ID":"c6c7a08a-8f34-41b0-99cd-9613bf032f67","Type":"ContainerStarted","Data":"bcb71c2de89f3d8dc0a898dd6f4235b6a48cc3778e0cb9bbe9d4e53c13e28aa6"}
Jul 17 09:28:46.431734 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.431702    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 17 09:28:46.453449 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.453418    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 17 09:28:46.453628 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.453569    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.456751 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.456727    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 17 09:28:46.456946 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.456931    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-j5qpm\""
Jul 17 09:28:46.460350 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.457323    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 17 09:28:46.460350 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.457349    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 17 09:28:46.460350 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.457511    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 17 09:28:46.460350 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.457558    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 17 09:28:46.460350 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.457736    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 17 09:28:46.460350 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.457791    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-9roqoskgrhhif\""
Jul 17 09:28:46.460350 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.458034    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 17 09:28:46.460350 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.458144    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 17 09:28:46.460350 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.458788    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 17 09:28:46.460350 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.459069    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 17 09:28:46.460350 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.459283    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 17 09:28:46.460350 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.460155    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 17 09:28:46.461790 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.461768    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 17 09:28:46.465201 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.464632    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 17 09:28:46.549330 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.549298    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/409e5645-8be5-4081-9459-e01da7957cfb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.549505 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.549338    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhfd8\" (UniqueName: \"kubernetes.io/projected/409e5645-8be5-4081-9459-e01da7957cfb-kube-api-access-rhfd8\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.549505 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.549374    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/409e5645-8be5-4081-9459-e01da7957cfb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.549505 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.549429    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/409e5645-8be5-4081-9459-e01da7957cfb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.549505 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.549470    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/409e5645-8be5-4081-9459-e01da7957cfb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.549722 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.549509    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/409e5645-8be5-4081-9459-e01da7957cfb-config\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.549722 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.549540    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/409e5645-8be5-4081-9459-e01da7957cfb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.549722 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.549574    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/409e5645-8be5-4081-9459-e01da7957cfb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.549722 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.549598    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/409e5645-8be5-4081-9459-e01da7957cfb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.549722 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.549640    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/409e5645-8be5-4081-9459-e01da7957cfb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.549982 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.549743    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/409e5645-8be5-4081-9459-e01da7957cfb-config-out\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.549982 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.549778    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/409e5645-8be5-4081-9459-e01da7957cfb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.549982 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.549808    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/409e5645-8be5-4081-9459-e01da7957cfb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.549982 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.549856    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/409e5645-8be5-4081-9459-e01da7957cfb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.549982 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.549894    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/409e5645-8be5-4081-9459-e01da7957cfb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.549982 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.549951    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/409e5645-8be5-4081-9459-e01da7957cfb-web-config\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.550315 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.549984    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/409e5645-8be5-4081-9459-e01da7957cfb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.550315 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.550013    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/409e5645-8be5-4081-9459-e01da7957cfb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.550315 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.550035    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/409e5645-8be5-4081-9459-e01da7957cfb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.550315 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.550078    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/409e5645-8be5-4081-9459-e01da7957cfb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.650639 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.650610    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/409e5645-8be5-4081-9459-e01da7957cfb-web-config\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.650639 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.650649    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/409e5645-8be5-4081-9459-e01da7957cfb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.650885 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.650679    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/409e5645-8be5-4081-9459-e01da7957cfb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.650885 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.650704    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/409e5645-8be5-4081-9459-e01da7957cfb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.650885 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.650743    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/409e5645-8be5-4081-9459-e01da7957cfb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.650885 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.650775    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/409e5645-8be5-4081-9459-e01da7957cfb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.650885 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.650801    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhfd8\" (UniqueName: \"kubernetes.io/projected/409e5645-8be5-4081-9459-e01da7957cfb-kube-api-access-rhfd8\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.650885 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.650835    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/409e5645-8be5-4081-9459-e01da7957cfb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.650885 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.650858    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/409e5645-8be5-4081-9459-e01da7957cfb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.651777 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.651746    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/409e5645-8be5-4081-9459-e01da7957cfb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.651940 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.651810    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/409e5645-8be5-4081-9459-e01da7957cfb-config\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.651940 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.651834    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/409e5645-8be5-4081-9459-e01da7957cfb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.651940 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.651829    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/409e5645-8be5-4081-9459-e01da7957cfb-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.651940 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.651869    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/409e5645-8be5-4081-9459-e01da7957cfb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.651940 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.651898    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/409e5645-8be5-4081-9459-e01da7957cfb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.651940 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.651921    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/409e5645-8be5-4081-9459-e01da7957cfb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.652365 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.651967    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/409e5645-8be5-4081-9459-e01da7957cfb-config-out\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.652365 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.651989    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/409e5645-8be5-4081-9459-e01da7957cfb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.652365 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.652016    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/409e5645-8be5-4081-9459-e01da7957cfb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.652365 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.652046    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/409e5645-8be5-4081-9459-e01da7957cfb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.652365 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.652077    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/409e5645-8be5-4081-9459-e01da7957cfb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.652365 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.652337    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/409e5645-8be5-4081-9459-e01da7957cfb-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.654626 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.654603    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/409e5645-8be5-4081-9459-e01da7957cfb-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.656886 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.656578    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/409e5645-8be5-4081-9459-e01da7957cfb-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.656886 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.656725    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/409e5645-8be5-4081-9459-e01da7957cfb-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.657059 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.657039    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/409e5645-8be5-4081-9459-e01da7957cfb-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.659420 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.659392    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/409e5645-8be5-4081-9459-e01da7957cfb-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.659970 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.659944    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/409e5645-8be5-4081-9459-e01da7957cfb-config\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.660076 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.659982    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/409e5645-8be5-4081-9459-e01da7957cfb-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.660525 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.660488    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/409e5645-8be5-4081-9459-e01da7957cfb-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.660647 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.660623    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/409e5645-8be5-4081-9459-e01da7957cfb-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.661137 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.661117    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/409e5645-8be5-4081-9459-e01da7957cfb-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.661408 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.661358    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/409e5645-8be5-4081-9459-e01da7957cfb-config-out\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.661523 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.661496    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/409e5645-8be5-4081-9459-e01da7957cfb-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.662176 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.662153    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/409e5645-8be5-4081-9459-e01da7957cfb-web-config\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.662723 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.662691    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/409e5645-8be5-4081-9459-e01da7957cfb-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.662860 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.662799    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/409e5645-8be5-4081-9459-e01da7957cfb-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.663344 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.663322    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/409e5645-8be5-4081-9459-e01da7957cfb-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.663805 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.663783    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/409e5645-8be5-4081-9459-e01da7957cfb-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.666070 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.665981    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhfd8\" (UniqueName: \"kubernetes.io/projected/409e5645-8be5-4081-9459-e01da7957cfb-kube-api-access-rhfd8\") pod \"prometheus-k8s-0\" (UID: \"409e5645-8be5-4081-9459-e01da7957cfb\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.771347 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.771287    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:28:46.930099 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:46.930068    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 17 09:28:47.110887 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:28:47.110825    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod409e5645_8be5_4081_9459_e01da7957cfb.slice/crio-bfe33599547dd62dd43efdf1ecb525b65cad7797927c4ffe06f50f28fd86c5f8 WatchSource:0}: Error finding container bfe33599547dd62dd43efdf1ecb525b65cad7797927c4ffe06f50f28fd86c5f8: Status 404 returned error can't find the container with id bfe33599547dd62dd43efdf1ecb525b65cad7797927c4ffe06f50f28fd86c5f8
Jul 17 09:28:47.427134 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:47.427098    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"409e5645-8be5-4081-9459-e01da7957cfb","Type":"ContainerStarted","Data":"bfe33599547dd62dd43efdf1ecb525b65cad7797927c4ffe06f50f28fd86c5f8"}
Jul 17 09:28:47.429056 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:47.429016    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-64874587b8-krgcd" event={"ID":"d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b","Type":"ContainerStarted","Data":"85513ef0642c8c8cc21b2c7dba8d8797df604848faf0e8a36eed192f295d8f29"}
Jul 17 09:28:47.446212 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:47.446164    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-64874587b8-krgcd" podStartSLOduration=1.3999398730000001 podStartE2EDuration="3.446148828s" totalImagesPullingTime="2.046208955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:28:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:47.445403279 +0000 UTC m=+125.028041655" watchObservedRunningTime="2026-07-17 09:28:47.446148828 +0000 UTC m=+125.028787203"
Jul 17 09:28:49.327341 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:49.327321    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:28:49.441519 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:49.441490    2574 generic.go:356] "Generic (PLEG): container finished" podID="409e5645-8be5-4081-9459-e01da7957cfb" containerID="23029a3df220ba040119296c97da401bb19269fe6fe7167cc17701033ee1e7dd" exitCode=0
Jul 17 09:28:49.441614 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:49.441578    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"409e5645-8be5-4081-9459-e01da7957cfb","Type":"ContainerDied","Data":"23029a3df220ba040119296c97da401bb19269fe6fe7167cc17701033ee1e7dd"}
Jul 17 09:28:49.443131 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:49.443103    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl" event={"ID":"c6c7a08a-8f34-41b0-99cd-9613bf032f67","Type":"ContainerStarted","Data":"d53812992d296f94c7d4f40d74e651ca816a6f9c753cbfee67dfed6263d3f7d5"}
Jul 17 09:28:50.449680 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:50.449650    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl" event={"ID":"c6c7a08a-8f34-41b0-99cd-9613bf032f67","Type":"ContainerStarted","Data":"bb4772513111354ff7eb9afb985f559c91dd235216d9f2c4c7e135b70f7bffe3"}
Jul 17 09:28:50.450096 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:50.449706    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl" event={"ID":"c6c7a08a-8f34-41b0-99cd-9613bf032f67","Type":"ContainerStarted","Data":"a1a61ddc3e2d5112a81034aada6d8e2dcd5969cf332eae088bf336aa1a26a84d"}
Jul 17 09:28:50.473207 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:50.473156    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6966f9675-gcrpl" podStartSLOduration=2.136054521 podStartE2EDuration="5.473137623s" totalImagesPullingTime="3.337083102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:28:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:50.470211825 +0000 UTC m=+128.052850202" watchObservedRunningTime="2026-07-17 09:28:50.473137623 +0000 UTC m=+128.055775998"
Jul 17 09:28:51.372183 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:51.372056    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-fdc6688f4-7rzg8"]
Jul 17 09:28:51.404739 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:51.404707    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fdc6688f4-7rzg8"]
Jul 17 09:28:51.404962 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:51.404831    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:28:51.493632 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:51.493598    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb78abf6-e9ea-49fa-ac96-87363bfd5974-service-ca\") pod \"console-fdc6688f4-7rzg8\" (UID: \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\") " pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:28:51.494118 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:51.493719    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb78abf6-e9ea-49fa-ac96-87363bfd5974-trusted-ca-bundle\") pod \"console-fdc6688f4-7rzg8\" (UID: \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\") " pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:28:51.494118 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:51.493756    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb78abf6-e9ea-49fa-ac96-87363bfd5974-console-serving-cert\") pod \"console-fdc6688f4-7rzg8\" (UID: \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\") " pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:28:51.494118 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:51.493781    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb78abf6-e9ea-49fa-ac96-87363bfd5974-console-oauth-config\") pod \"console-fdc6688f4-7rzg8\" (UID: \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\") " pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:28:51.494118 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:51.493798    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqcbz\" (UniqueName: \"kubernetes.io/projected/fb78abf6-e9ea-49fa-ac96-87363bfd5974-kube-api-access-lqcbz\") pod \"console-fdc6688f4-7rzg8\" (UID: \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\") " pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:28:51.494118 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:51.493883    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb78abf6-e9ea-49fa-ac96-87363bfd5974-console-config\") pod \"console-fdc6688f4-7rzg8\" (UID: \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\") " pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:28:51.494118 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:51.493917    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb78abf6-e9ea-49fa-ac96-87363bfd5974-oauth-serving-cert\") pod \"console-fdc6688f4-7rzg8\" (UID: \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\") " pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:28:51.596877 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:51.595070    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb78abf6-e9ea-49fa-ac96-87363bfd5974-trusted-ca-bundle\") pod \"console-fdc6688f4-7rzg8\" (UID: \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\") " pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:28:51.596877 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:51.595119    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb78abf6-e9ea-49fa-ac96-87363bfd5974-console-serving-cert\") pod \"console-fdc6688f4-7rzg8\" (UID: \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\") " pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:28:51.596877 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:51.595151    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb78abf6-e9ea-49fa-ac96-87363bfd5974-console-oauth-config\") pod \"console-fdc6688f4-7rzg8\" (UID: \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\") " pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:28:51.596877 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:51.595173    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqcbz\" (UniqueName: \"kubernetes.io/projected/fb78abf6-e9ea-49fa-ac96-87363bfd5974-kube-api-access-lqcbz\") pod \"console-fdc6688f4-7rzg8\" (UID: \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\") " pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:28:51.596877 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:51.595218    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb78abf6-e9ea-49fa-ac96-87363bfd5974-console-config\") pod \"console-fdc6688f4-7rzg8\" (UID: \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\") " pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:28:51.596877 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:51.595269    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb78abf6-e9ea-49fa-ac96-87363bfd5974-oauth-serving-cert\") pod \"console-fdc6688f4-7rzg8\" (UID: \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\") " pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:28:51.596877 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:51.595365    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb78abf6-e9ea-49fa-ac96-87363bfd5974-service-ca\") pod \"console-fdc6688f4-7rzg8\" (UID: \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\") " pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:28:51.596877 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:51.596088    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb78abf6-e9ea-49fa-ac96-87363bfd5974-service-ca\") pod \"console-fdc6688f4-7rzg8\" (UID: \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\") " pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:28:51.598139 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:51.598115    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb78abf6-e9ea-49fa-ac96-87363bfd5974-console-config\") pod \"console-fdc6688f4-7rzg8\" (UID: \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\") " pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:28:51.598304 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:51.598114    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb78abf6-e9ea-49fa-ac96-87363bfd5974-trusted-ca-bundle\") pod \"console-fdc6688f4-7rzg8\" (UID: \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\") " pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:28:51.598304 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:51.598153    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb78abf6-e9ea-49fa-ac96-87363bfd5974-oauth-serving-cert\") pod \"console-fdc6688f4-7rzg8\" (UID: \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\") " pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:28:51.600206 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:51.599943    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb78abf6-e9ea-49fa-ac96-87363bfd5974-console-serving-cert\") pod \"console-fdc6688f4-7rzg8\" (UID: \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\") " pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:28:51.600758 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:51.600737    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb78abf6-e9ea-49fa-ac96-87363bfd5974-console-oauth-config\") pod \"console-fdc6688f4-7rzg8\" (UID: \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\") " pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:28:51.604304 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:51.604263    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqcbz\" (UniqueName: \"kubernetes.io/projected/fb78abf6-e9ea-49fa-ac96-87363bfd5974-kube-api-access-lqcbz\") pod \"console-fdc6688f4-7rzg8\" (UID: \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\") " pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:28:51.719955 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:51.719909    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:28:52.606080 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:52.606042    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb87a089-9075-4b84-a8b6-98cfaa1d9387-metrics-certs\") pod \"network-metrics-daemon-s62sx\" (UID: \"fb87a089-9075-4b84-a8b6-98cfaa1d9387\") " pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:28:52.609758 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:52.609722    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fb87a089-9075-4b84-a8b6-98cfaa1d9387-metrics-certs\") pod \"network-metrics-daemon-s62sx\" (UID: \"fb87a089-9075-4b84-a8b6-98cfaa1d9387\") " pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:28:52.675318 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:52.675288    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-fdc6688f4-7rzg8"]
Jul 17 09:28:52.678735 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:28:52.678713    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb78abf6_e9ea_49fa_ac96_87363bfd5974.slice/crio-4b4c7c1f58c578e3ec194fa6399d783a8467406189c6b2e400857e283fb3dd6f WatchSource:0}: Error finding container 4b4c7c1f58c578e3ec194fa6399d783a8467406189c6b2e400857e283fb3dd6f: Status 404 returned error can't find the container with id 4b4c7c1f58c578e3ec194fa6399d783a8467406189c6b2e400857e283fb3dd6f
Jul 17 09:28:52.819017 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:52.818958    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jxtjz\""
Jul 17 09:28:52.827202 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:52.827179    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-s62sx"
Jul 17 09:28:53.090363 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:53.090337    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-s62sx"]
Jul 17 09:28:53.092534 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:28:53.092508    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb87a089_9075_4b84_a8b6_98cfaa1d9387.slice/crio-aa25ea2a583992818c8d67a8f798bd146e242174faa86864a15225848c287ec7 WatchSource:0}: Error finding container aa25ea2a583992818c8d67a8f798bd146e242174faa86864a15225848c287ec7: Status 404 returned error can't find the container with id aa25ea2a583992818c8d67a8f798bd146e242174faa86864a15225848c287ec7
Jul 17 09:28:53.466087 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:53.466052    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s62sx" event={"ID":"fb87a089-9075-4b84-a8b6-98cfaa1d9387","Type":"ContainerStarted","Data":"aa25ea2a583992818c8d67a8f798bd146e242174faa86864a15225848c287ec7"}
Jul 17 09:28:53.468084 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:53.468055    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"409e5645-8be5-4081-9459-e01da7957cfb","Type":"ContainerStarted","Data":"88da2c640962861d4e75567f000337417335157c34d319de0edc7686763853a9"}
Jul 17 09:28:53.468201 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:53.468128    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"409e5645-8be5-4081-9459-e01da7957cfb","Type":"ContainerStarted","Data":"3a4a2f99307af6a6cb4de8dfb5fd9485029d1737b1a1ccfa6a84a7c4c5fe171f"}
Jul 17 09:28:53.469380 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:53.469354    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fdc6688f4-7rzg8" event={"ID":"fb78abf6-e9ea-49fa-ac96-87363bfd5974","Type":"ContainerStarted","Data":"873bfa7b7f86e1e798707964cf77e5d357416aee6ae00236c25ec8803c2eab55"}
Jul 17 09:28:53.469450 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:53.469400    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fdc6688f4-7rzg8" event={"ID":"fb78abf6-e9ea-49fa-ac96-87363bfd5974","Type":"ContainerStarted","Data":"4b4c7c1f58c578e3ec194fa6399d783a8467406189c6b2e400857e283fb3dd6f"}
Jul 17 09:28:53.490869 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:53.490827    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-fdc6688f4-7rzg8" podStartSLOduration=2.490813537 podStartE2EDuration="2.490813537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:28:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:53.488602586 +0000 UTC m=+131.071240982" watchObservedRunningTime="2026-07-17 09:28:53.490813537 +0000 UTC m=+131.073451913"
Jul 17 09:28:54.350355 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:54.350298    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-75c884f5f7-7k78v" podUID="27c6d907-d449-4202-8700-dc369dadb9f4" containerName="registry" containerID="cri-o://f2edd97a983faffb6e379bfce59d145c8f25a8fc224454f35f5c8358e2f43710" gracePeriod=30
Jul 17 09:28:54.677965 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:54.677832    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:28:54.726936 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:54.726804    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/27c6d907-d449-4202-8700-dc369dadb9f4-registry-certificates\") pod \"27c6d907-d449-4202-8700-dc369dadb9f4\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") "
Jul 17 09:28:54.726936 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:54.726860    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/27c6d907-d449-4202-8700-dc369dadb9f4-image-registry-private-configuration\") pod \"27c6d907-d449-4202-8700-dc369dadb9f4\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") "
Jul 17 09:28:54.726936 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:54.726896    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-bound-sa-token\") pod \"27c6d907-d449-4202-8700-dc369dadb9f4\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") "
Jul 17 09:28:54.726936 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:54.726931    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-registry-tls\") pod \"27c6d907-d449-4202-8700-dc369dadb9f4\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") "
Jul 17 09:28:54.727229 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:54.726997    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/27c6d907-d449-4202-8700-dc369dadb9f4-trusted-ca\") pod \"27c6d907-d449-4202-8700-dc369dadb9f4\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") "
Jul 17 09:28:54.727229 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:54.727034    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kj6hb\" (UniqueName: \"kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-kube-api-access-kj6hb\") pod \"27c6d907-d449-4202-8700-dc369dadb9f4\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") "
Jul 17 09:28:54.727229 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:54.727060    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/27c6d907-d449-4202-8700-dc369dadb9f4-ca-trust-extracted\") pod \"27c6d907-d449-4202-8700-dc369dadb9f4\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") "
Jul 17 09:28:54.727229 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:54.727122    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/27c6d907-d449-4202-8700-dc369dadb9f4-installation-pull-secrets\") pod \"27c6d907-d449-4202-8700-dc369dadb9f4\" (UID: \"27c6d907-d449-4202-8700-dc369dadb9f4\") "
Jul 17 09:28:54.727694 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:54.727647    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27c6d907-d449-4202-8700-dc369dadb9f4-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "27c6d907-d449-4202-8700-dc369dadb9f4" (UID: "27c6d907-d449-4202-8700-dc369dadb9f4"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:28:54.727694 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:54.727673    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27c6d907-d449-4202-8700-dc369dadb9f4-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "27c6d907-d449-4202-8700-dc369dadb9f4" (UID: "27c6d907-d449-4202-8700-dc369dadb9f4"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:28:54.732859 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:54.732834    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "27c6d907-d449-4202-8700-dc369dadb9f4" (UID: "27c6d907-d449-4202-8700-dc369dadb9f4"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:28:54.733631 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:54.733606    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "27c6d907-d449-4202-8700-dc369dadb9f4" (UID: "27c6d907-d449-4202-8700-dc369dadb9f4"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:28:54.733728 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:54.733708    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27c6d907-d449-4202-8700-dc369dadb9f4-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "27c6d907-d449-4202-8700-dc369dadb9f4" (UID: "27c6d907-d449-4202-8700-dc369dadb9f4"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:28:54.733793 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:54.733724    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-kube-api-access-kj6hb" (OuterVolumeSpecName: "kube-api-access-kj6hb") pod "27c6d907-d449-4202-8700-dc369dadb9f4" (UID: "27c6d907-d449-4202-8700-dc369dadb9f4"). InnerVolumeSpecName "kube-api-access-kj6hb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:28:54.735026 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:54.735001    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27c6d907-d449-4202-8700-dc369dadb9f4-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "27c6d907-d449-4202-8700-dc369dadb9f4" (UID: "27c6d907-d449-4202-8700-dc369dadb9f4"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:28:54.739099 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:54.739070    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/27c6d907-d449-4202-8700-dc369dadb9f4-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "27c6d907-d449-4202-8700-dc369dadb9f4" (UID: "27c6d907-d449-4202-8700-dc369dadb9f4"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:28:54.828105 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:54.828082    2574 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-bound-sa-token\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:28:54.828233 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:54.828109    2574 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-registry-tls\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:28:54.828233 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:54.828122    2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/27c6d907-d449-4202-8700-dc369dadb9f4-trusted-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:28:54.828233 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:54.828135    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kj6hb\" (UniqueName: \"kubernetes.io/projected/27c6d907-d449-4202-8700-dc369dadb9f4-kube-api-access-kj6hb\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:28:54.828233 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:54.828147    2574 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/27c6d907-d449-4202-8700-dc369dadb9f4-ca-trust-extracted\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:28:54.828233 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:54.828159    2574 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/27c6d907-d449-4202-8700-dc369dadb9f4-installation-pull-secrets\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:28:54.828233 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:54.828171    2574 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/27c6d907-d449-4202-8700-dc369dadb9f4-registry-certificates\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:28:54.828233 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:54.828185    2574 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/27c6d907-d449-4202-8700-dc369dadb9f4-image-registry-private-configuration\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:28:55.480918 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:55.480881    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s62sx" event={"ID":"fb87a089-9075-4b84-a8b6-98cfaa1d9387","Type":"ContainerStarted","Data":"9c3424cc1d2308c9797b44eb87201def7e379f05ef5a66ea288f4116af9450ea"}
Jul 17 09:28:55.481415 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:55.480945    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-s62sx" event={"ID":"fb87a089-9075-4b84-a8b6-98cfaa1d9387","Type":"ContainerStarted","Data":"8523b55d7d40add84315f1cd2ae25cb3f519f9f4b4648063ea064b15c669fdf3"}
Jul 17 09:28:55.482381 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:55.482353    2574 generic.go:356] "Generic (PLEG): container finished" podID="27c6d907-d449-4202-8700-dc369dadb9f4" containerID="f2edd97a983faffb6e379bfce59d145c8f25a8fc224454f35f5c8358e2f43710" exitCode=0
Jul 17 09:28:55.482498 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:55.482396    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75c884f5f7-7k78v"
Jul 17 09:28:55.482498 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:55.482433    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75c884f5f7-7k78v" event={"ID":"27c6d907-d449-4202-8700-dc369dadb9f4","Type":"ContainerDied","Data":"f2edd97a983faffb6e379bfce59d145c8f25a8fc224454f35f5c8358e2f43710"}
Jul 17 09:28:55.482498 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:55.482481    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75c884f5f7-7k78v" event={"ID":"27c6d907-d449-4202-8700-dc369dadb9f4","Type":"ContainerDied","Data":"603670d0b965930292f421166c387325105ce5a5cf8b012f621a76c21934ccb1"}
Jul 17 09:28:55.482653 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:55.482509    2574 scope.go:117] "RemoveContainer" containerID="f2edd97a983faffb6e379bfce59d145c8f25a8fc224454f35f5c8358e2f43710"
Jul 17 09:28:55.495606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:55.495538    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-s62sx" podStartSLOduration=131.095199144 podStartE2EDuration="2m12.495525431s" totalImagesPullingTime="1.400326287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:26:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:55.495361455 +0000 UTC m=+133.077999831" watchObservedRunningTime="2026-07-17 09:28:55.495525431 +0000 UTC m=+133.078163807"
Jul 17 09:28:55.511534 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:55.511513    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-75c884f5f7-7k78v"]
Jul 17 09:28:55.514553 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:55.514528    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-75c884f5f7-7k78v"]
Jul 17 09:28:56.024253 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:56.024215    2574 scope.go:117] "RemoveContainer" containerID="f2edd97a983faffb6e379bfce59d145c8f25a8fc224454f35f5c8358e2f43710"
Jul 17 09:28:56.024612 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:28:56.024588    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2edd97a983faffb6e379bfce59d145c8f25a8fc224454f35f5c8358e2f43710\": container with ID starting with f2edd97a983faffb6e379bfce59d145c8f25a8fc224454f35f5c8358e2f43710 not found: ID does not exist" containerID="f2edd97a983faffb6e379bfce59d145c8f25a8fc224454f35f5c8358e2f43710"
Jul 17 09:28:56.024664 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:56.024625    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2edd97a983faffb6e379bfce59d145c8f25a8fc224454f35f5c8358e2f43710"} err="failed to get container status \"f2edd97a983faffb6e379bfce59d145c8f25a8fc224454f35f5c8358e2f43710\": rpc error: code = NotFound desc = could not find container \"f2edd97a983faffb6e379bfce59d145c8f25a8fc224454f35f5c8358e2f43710\": container with ID starting with f2edd97a983faffb6e379bfce59d145c8f25a8fc224454f35f5c8358e2f43710 not found: ID does not exist"
Jul 17 09:28:56.352819 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:56.352731    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-689d766b54-g8vpn" podUID="1b1d64eb-7388-44f0-8600-2dcbea6896c7" containerName="console" containerID="cri-o://d9b3193cf6c9090b458e0b4fa8282c73d40adbc3df691b4d2b58312655805d7a" gracePeriod=15
Jul 17 09:28:56.488523 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:56.488494    2574 generic.go:356] "Generic (PLEG): container finished" podID="1b1d64eb-7388-44f0-8600-2dcbea6896c7" containerID="d9b3193cf6c9090b458e0b4fa8282c73d40adbc3df691b4d2b58312655805d7a" exitCode=0
Jul 17 09:28:56.488861 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:56.488573    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-689d766b54-g8vpn" event={"ID":"1b1d64eb-7388-44f0-8600-2dcbea6896c7","Type":"ContainerDied","Data":"d9b3193cf6c9090b458e0b4fa8282c73d40adbc3df691b4d2b58312655805d7a"}
Jul 17 09:28:56.671143 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:56.671101    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-689d766b54-g8vpn"
Jul 17 09:28:56.745277 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:56.745216    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b1d64eb-7388-44f0-8600-2dcbea6896c7-console-oauth-config\") pod \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\" (UID: \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\") "
Jul 17 09:28:56.745384 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:56.745290    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b1d64eb-7388-44f0-8600-2dcbea6896c7-oauth-serving-cert\") pod \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\" (UID: \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\") "
Jul 17 09:28:56.745461 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:56.745444    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72wfd\" (UniqueName: \"kubernetes.io/projected/1b1d64eb-7388-44f0-8600-2dcbea6896c7-kube-api-access-72wfd\") pod \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\" (UID: \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\") "
Jul 17 09:28:56.745517 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:56.745480    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b1d64eb-7388-44f0-8600-2dcbea6896c7-console-config\") pod \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\" (UID: \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\") "
Jul 17 09:28:56.745570 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:56.745527    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b1d64eb-7388-44f0-8600-2dcbea6896c7-service-ca\") pod \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\" (UID: \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\") "
Jul 17 09:28:56.745570 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:56.745553    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b1d64eb-7388-44f0-8600-2dcbea6896c7-console-serving-cert\") pod \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\" (UID: \"1b1d64eb-7388-44f0-8600-2dcbea6896c7\") "
Jul 17 09:28:56.745685 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:56.745659    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b1d64eb-7388-44f0-8600-2dcbea6896c7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1b1d64eb-7388-44f0-8600-2dcbea6896c7" (UID: "1b1d64eb-7388-44f0-8600-2dcbea6896c7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:28:56.746040 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:56.745847    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b1d64eb-7388-44f0-8600-2dcbea6896c7-console-config" (OuterVolumeSpecName: "console-config") pod "1b1d64eb-7388-44f0-8600-2dcbea6896c7" (UID: "1b1d64eb-7388-44f0-8600-2dcbea6896c7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:28:56.746040 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:56.746012    2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b1d64eb-7388-44f0-8600-2dcbea6896c7-console-config\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:28:56.746040 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:56.746030    2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b1d64eb-7388-44f0-8600-2dcbea6896c7-oauth-serving-cert\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:28:56.746375 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:56.746352    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b1d64eb-7388-44f0-8600-2dcbea6896c7-service-ca" (OuterVolumeSpecName: "service-ca") pod "1b1d64eb-7388-44f0-8600-2dcbea6896c7" (UID: "1b1d64eb-7388-44f0-8600-2dcbea6896c7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:28:56.747935 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:56.747915    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b1d64eb-7388-44f0-8600-2dcbea6896c7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1b1d64eb-7388-44f0-8600-2dcbea6896c7" (UID: "1b1d64eb-7388-44f0-8600-2dcbea6896c7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:28:56.748017 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:56.747997    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b1d64eb-7388-44f0-8600-2dcbea6896c7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1b1d64eb-7388-44f0-8600-2dcbea6896c7" (UID: "1b1d64eb-7388-44f0-8600-2dcbea6896c7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:28:56.748166 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:56.748149    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b1d64eb-7388-44f0-8600-2dcbea6896c7-kube-api-access-72wfd" (OuterVolumeSpecName: "kube-api-access-72wfd") pod "1b1d64eb-7388-44f0-8600-2dcbea6896c7" (UID: "1b1d64eb-7388-44f0-8600-2dcbea6896c7"). InnerVolumeSpecName "kube-api-access-72wfd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:28:56.847173 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:56.847152    2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b1d64eb-7388-44f0-8600-2dcbea6896c7-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:28:56.847296 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:56.847177    2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b1d64eb-7388-44f0-8600-2dcbea6896c7-console-serving-cert\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:28:56.847296 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:56.847191    2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b1d64eb-7388-44f0-8600-2dcbea6896c7-console-oauth-config\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:28:56.847296 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:56.847203    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72wfd\" (UniqueName: \"kubernetes.io/projected/1b1d64eb-7388-44f0-8600-2dcbea6896c7-kube-api-access-72wfd\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:28:56.897932 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:56.897906    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27c6d907-d449-4202-8700-dc369dadb9f4" path="/var/lib/kubelet/pods/27c6d907-d449-4202-8700-dc369dadb9f4/volumes"
Jul 17 09:28:57.496989 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:57.496957    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"409e5645-8be5-4081-9459-e01da7957cfb","Type":"ContainerStarted","Data":"8699e38f97f8f13b1b0a1eb2e5aae266bd0d761aaa653bd21f3e34f045d52af5"}
Jul 17 09:28:57.497400 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:57.497040    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"409e5645-8be5-4081-9459-e01da7957cfb","Type":"ContainerStarted","Data":"6e5ecb8d462d32918a9389048749d78af686ddf2a2b7e9c393dc2e1e2bc28848"}
Jul 17 09:28:57.497400 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:57.497070    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"409e5645-8be5-4081-9459-e01da7957cfb","Type":"ContainerStarted","Data":"b26222e99a7bc7a68544afbd49bf61b655900635232e76fe18901154f9b8032d"}
Jul 17 09:28:57.497400 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:57.497103    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"409e5645-8be5-4081-9459-e01da7957cfb","Type":"ContainerStarted","Data":"bed58bdff7bb1e1296382e5022cb2364a33f50b8a5f6f66cdeefce52f6ce3a7d"}
Jul 17 09:28:57.498289 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:57.498270    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-689d766b54-g8vpn" event={"ID":"1b1d64eb-7388-44f0-8600-2dcbea6896c7","Type":"ContainerDied","Data":"125bbe8ec24cd0e2628f02e2ff52129ee039666f3cd62c68a1b60bf2cea6f392"}
Jul 17 09:28:57.498361 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:57.498295    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-689d766b54-g8vpn"
Jul 17 09:28:57.498361 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:57.498322    2574 scope.go:117] "RemoveContainer" containerID="d9b3193cf6c9090b458e0b4fa8282c73d40adbc3df691b4d2b58312655805d7a"
Jul 17 09:28:57.526007 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:57.525961    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.7192696400000003 podStartE2EDuration="11.525943743s" totalImagesPullingTime="8.774591069s" totalInitContainerRuntime="32.083034ms" isStatefulPod=false podCreationTimestamp="2026-07-17 09:28:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:28:57.524619352 +0000 UTC m=+135.107257729" watchObservedRunningTime="2026-07-17 09:28:57.525943743 +0000 UTC m=+135.108582120"
Jul 17 09:28:57.537157 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:57.537131    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-689d766b54-g8vpn"]
Jul 17 09:28:57.540509 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:57.540489    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-689d766b54-g8vpn"]
Jul 17 09:28:58.897094 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:28:58.897062    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b1d64eb-7388-44f0-8600-2dcbea6896c7" path="/var/lib/kubelet/pods/1b1d64eb-7388-44f0-8600-2dcbea6896c7/volumes"
Jul 17 09:29:01.720462 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:01.720435    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:29:01.720833 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:01.720519    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:29:01.724976 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:01.724958    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:29:01.772166 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:01.772148    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:29:02.522173 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:02.522147    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:29:02.571359 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:02.571335    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56f98c7866-ww6gm"]
Jul 17 09:29:04.921291 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:04.921262    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-64874587b8-krgcd"
Jul 17 09:29:04.921694 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:04.921309    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-64874587b8-krgcd"
Jul 17 09:29:08.546205 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:08.546177    2574 generic.go:356] "Generic (PLEG): container finished" podID="1931d80b-6799-4456-a479-23d203f2070a" containerID="a0ac9d069c9cad7b5bdfbc4a678ab6abd77501558eae3e488d8dde9f4c6ae56f" exitCode=0
Jul 17 09:29:08.546590 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:08.546259    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jxtbz" event={"ID":"1931d80b-6799-4456-a479-23d203f2070a","Type":"ContainerDied","Data":"a0ac9d069c9cad7b5bdfbc4a678ab6abd77501558eae3e488d8dde9f4c6ae56f"}
Jul 17 09:29:08.546590 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:08.546527    2574 scope.go:117] "RemoveContainer" containerID="a0ac9d069c9cad7b5bdfbc4a678ab6abd77501558eae3e488d8dde9f4c6ae56f"
Jul 17 09:29:09.554681 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:09.554641    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-jxtbz" event={"ID":"1931d80b-6799-4456-a479-23d203f2070a","Type":"ContainerStarted","Data":"4710a053a8c6fccf41f29334138aefd30dbf2ae3ac854b7eb0d64268b043769c"}
Jul 17 09:29:24.926199 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:24.926173    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-64874587b8-krgcd"
Jul 17 09:29:24.930276 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:24.930228    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-64874587b8-krgcd"
Jul 17 09:29:27.593134 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:27.593067    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-56f98c7866-ww6gm" podUID="5a83a14c-2f3b-4482-8557-131da3e63529" containerName="console" containerID="cri-o://282fd27fd6e850c85709ef1c081219d0275384a8101ef1af2bc50b4fbf4e170c" gracePeriod=15
Jul 17 09:29:28.332493 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.332470    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:29:28.486616 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.486550    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9l8xc\" (UniqueName: \"kubernetes.io/projected/5a83a14c-2f3b-4482-8557-131da3e63529-kube-api-access-9l8xc\") pod \"5a83a14c-2f3b-4482-8557-131da3e63529\" (UID: \"5a83a14c-2f3b-4482-8557-131da3e63529\") "
Jul 17 09:29:28.486616 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.486605    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a83a14c-2f3b-4482-8557-131da3e63529-oauth-serving-cert\") pod \"5a83a14c-2f3b-4482-8557-131da3e63529\" (UID: \"5a83a14c-2f3b-4482-8557-131da3e63529\") "
Jul 17 09:29:28.486767 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.486627    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a83a14c-2f3b-4482-8557-131da3e63529-trusted-ca-bundle\") pod \"5a83a14c-2f3b-4482-8557-131da3e63529\" (UID: \"5a83a14c-2f3b-4482-8557-131da3e63529\") "
Jul 17 09:29:28.486767 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.486744    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a83a14c-2f3b-4482-8557-131da3e63529-console-oauth-config\") pod \"5a83a14c-2f3b-4482-8557-131da3e63529\" (UID: \"5a83a14c-2f3b-4482-8557-131da3e63529\") "
Jul 17 09:29:28.486878 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.486837    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a83a14c-2f3b-4482-8557-131da3e63529-service-ca\") pod \"5a83a14c-2f3b-4482-8557-131da3e63529\" (UID: \"5a83a14c-2f3b-4482-8557-131da3e63529\") "
Jul 17 09:29:28.486878 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.486869    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a83a14c-2f3b-4482-8557-131da3e63529-console-serving-cert\") pod \"5a83a14c-2f3b-4482-8557-131da3e63529\" (UID: \"5a83a14c-2f3b-4482-8557-131da3e63529\") "
Jul 17 09:29:28.486983 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.486898    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a83a14c-2f3b-4482-8557-131da3e63529-console-config\") pod \"5a83a14c-2f3b-4482-8557-131da3e63529\" (UID: \"5a83a14c-2f3b-4482-8557-131da3e63529\") "
Jul 17 09:29:28.486983 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.486970    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a83a14c-2f3b-4482-8557-131da3e63529-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5a83a14c-2f3b-4482-8557-131da3e63529" (UID: "5a83a14c-2f3b-4482-8557-131da3e63529"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:29:28.487081 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.486997    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a83a14c-2f3b-4482-8557-131da3e63529-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5a83a14c-2f3b-4482-8557-131da3e63529" (UID: "5a83a14c-2f3b-4482-8557-131da3e63529"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:29:28.487180 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.487160    2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a83a14c-2f3b-4482-8557-131da3e63529-oauth-serving-cert\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:29:28.487254 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.487185    2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a83a14c-2f3b-4482-8557-131da3e63529-trusted-ca-bundle\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:29:28.487315 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.487265    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a83a14c-2f3b-4482-8557-131da3e63529-service-ca" (OuterVolumeSpecName: "service-ca") pod "5a83a14c-2f3b-4482-8557-131da3e63529" (UID: "5a83a14c-2f3b-4482-8557-131da3e63529"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:29:28.487535 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.487422    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a83a14c-2f3b-4482-8557-131da3e63529-console-config" (OuterVolumeSpecName: "console-config") pod "5a83a14c-2f3b-4482-8557-131da3e63529" (UID: "5a83a14c-2f3b-4482-8557-131da3e63529"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:29:28.488777 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.488756    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a83a14c-2f3b-4482-8557-131da3e63529-kube-api-access-9l8xc" (OuterVolumeSpecName: "kube-api-access-9l8xc") pod "5a83a14c-2f3b-4482-8557-131da3e63529" (UID: "5a83a14c-2f3b-4482-8557-131da3e63529"). InnerVolumeSpecName "kube-api-access-9l8xc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:29:28.488875 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.488822    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a83a14c-2f3b-4482-8557-131da3e63529-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5a83a14c-2f3b-4482-8557-131da3e63529" (UID: "5a83a14c-2f3b-4482-8557-131da3e63529"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:29:28.488875 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.488862    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5a83a14c-2f3b-4482-8557-131da3e63529-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5a83a14c-2f3b-4482-8557-131da3e63529" (UID: "5a83a14c-2f3b-4482-8557-131da3e63529"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:29:28.588362 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.588339    2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a83a14c-2f3b-4482-8557-131da3e63529-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:29:28.588362 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.588357    2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a83a14c-2f3b-4482-8557-131da3e63529-console-serving-cert\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:29:28.588491 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.588367    2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a83a14c-2f3b-4482-8557-131da3e63529-console-config\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:29:28.588491 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.588376    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9l8xc\" (UniqueName: \"kubernetes.io/projected/5a83a14c-2f3b-4482-8557-131da3e63529-kube-api-access-9l8xc\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:29:28.588491 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.588384    2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a83a14c-2f3b-4482-8557-131da3e63529-console-oauth-config\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:29:28.633850 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.633825    2574 generic.go:356] "Generic (PLEG): container finished" podID="5a83a14c-2f3b-4482-8557-131da3e63529" containerID="282fd27fd6e850c85709ef1c081219d0275384a8101ef1af2bc50b4fbf4e170c" exitCode=0
Jul 17 09:29:28.634178 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.633881    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56f98c7866-ww6gm"
Jul 17 09:29:28.634178 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.633895    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f98c7866-ww6gm" event={"ID":"5a83a14c-2f3b-4482-8557-131da3e63529","Type":"ContainerDied","Data":"282fd27fd6e850c85709ef1c081219d0275384a8101ef1af2bc50b4fbf4e170c"}
Jul 17 09:29:28.634178 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.633936    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56f98c7866-ww6gm" event={"ID":"5a83a14c-2f3b-4482-8557-131da3e63529","Type":"ContainerDied","Data":"3eb06f794f6802f016ce4afee9ef85a7892e6b329cc67ad6c234d2e8d1f7a49c"}
Jul 17 09:29:28.634178 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.633954    2574 scope.go:117] "RemoveContainer" containerID="282fd27fd6e850c85709ef1c081219d0275384a8101ef1af2bc50b4fbf4e170c"
Jul 17 09:29:28.642140 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.642126    2574 scope.go:117] "RemoveContainer" containerID="282fd27fd6e850c85709ef1c081219d0275384a8101ef1af2bc50b4fbf4e170c"
Jul 17 09:29:28.642380 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:29:28.642364    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"282fd27fd6e850c85709ef1c081219d0275384a8101ef1af2bc50b4fbf4e170c\": container with ID starting with 282fd27fd6e850c85709ef1c081219d0275384a8101ef1af2bc50b4fbf4e170c not found: ID does not exist" containerID="282fd27fd6e850c85709ef1c081219d0275384a8101ef1af2bc50b4fbf4e170c"
Jul 17 09:29:28.642434 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.642386    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"282fd27fd6e850c85709ef1c081219d0275384a8101ef1af2bc50b4fbf4e170c"} err="failed to get container status \"282fd27fd6e850c85709ef1c081219d0275384a8101ef1af2bc50b4fbf4e170c\": rpc error: code = NotFound desc = could not find container \"282fd27fd6e850c85709ef1c081219d0275384a8101ef1af2bc50b4fbf4e170c\": container with ID starting with 282fd27fd6e850c85709ef1c081219d0275384a8101ef1af2bc50b4fbf4e170c not found: ID does not exist"
Jul 17 09:29:28.654671 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.654645    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56f98c7866-ww6gm"]
Jul 17 09:29:28.658641 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.658619    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56f98c7866-ww6gm"]
Jul 17 09:29:28.895668 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:28.895616    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a83a14c-2f3b-4482-8557-131da3e63529" path="/var/lib/kubelet/pods/5a83a14c-2f3b-4482-8557-131da3e63529/volumes"
Jul 17 09:29:46.771828 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:46.771794    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:29:46.791372 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:46.791339    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:29:47.718537 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:29:47.718511    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 17 09:30:11.468472 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:11.468438    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fdc6688f4-7rzg8"]
Jul 17 09:30:36.489075 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.489027    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-fdc6688f4-7rzg8" podUID="fb78abf6-e9ea-49fa-ac96-87363bfd5974" containerName="console" containerID="cri-o://873bfa7b7f86e1e798707964cf77e5d357416aee6ae00236c25ec8803c2eab55" gracePeriod=15
Jul 17 09:30:36.742421 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.742356    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:30:36.887047 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.887009    2574 generic.go:356] "Generic (PLEG): container finished" podID="fb78abf6-e9ea-49fa-ac96-87363bfd5974" containerID="873bfa7b7f86e1e798707964cf77e5d357416aee6ae00236c25ec8803c2eab55" exitCode=0
Jul 17 09:30:36.887047 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.887053    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-fdc6688f4-7rzg8"
Jul 17 09:30:36.887270 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.887095    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fdc6688f4-7rzg8" event={"ID":"fb78abf6-e9ea-49fa-ac96-87363bfd5974","Type":"ContainerDied","Data":"873bfa7b7f86e1e798707964cf77e5d357416aee6ae00236c25ec8803c2eab55"}
Jul 17 09:30:36.887270 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.887171    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-fdc6688f4-7rzg8" event={"ID":"fb78abf6-e9ea-49fa-ac96-87363bfd5974","Type":"ContainerDied","Data":"4b4c7c1f58c578e3ec194fa6399d783a8467406189c6b2e400857e283fb3dd6f"}
Jul 17 09:30:36.887270 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.887194    2574 scope.go:117] "RemoveContainer" containerID="873bfa7b7f86e1e798707964cf77e5d357416aee6ae00236c25ec8803c2eab55"
Jul 17 09:30:36.892287 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.892261    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb78abf6-e9ea-49fa-ac96-87363bfd5974-service-ca\") pod \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\" (UID: \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\") "
Jul 17 09:30:36.892408 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.892307    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb78abf6-e9ea-49fa-ac96-87363bfd5974-console-config\") pod \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\" (UID: \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\") "
Jul 17 09:30:36.892408 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.892343    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb78abf6-e9ea-49fa-ac96-87363bfd5974-console-oauth-config\") pod \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\" (UID: \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\") "
Jul 17 09:30:36.892408 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.892368    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqcbz\" (UniqueName: \"kubernetes.io/projected/fb78abf6-e9ea-49fa-ac96-87363bfd5974-kube-api-access-lqcbz\") pod \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\" (UID: \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\") "
Jul 17 09:30:36.892408 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.892404    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb78abf6-e9ea-49fa-ac96-87363bfd5974-oauth-serving-cert\") pod \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\" (UID: \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\") "
Jul 17 09:30:36.892609 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.892540    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb78abf6-e9ea-49fa-ac96-87363bfd5974-trusted-ca-bundle\") pod \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\" (UID: \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\") "
Jul 17 09:30:36.892609 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.892592    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb78abf6-e9ea-49fa-ac96-87363bfd5974-console-serving-cert\") pod \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\" (UID: \"fb78abf6-e9ea-49fa-ac96-87363bfd5974\") "
Jul 17 09:30:36.892808 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.892676    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb78abf6-e9ea-49fa-ac96-87363bfd5974-service-ca" (OuterVolumeSpecName: "service-ca") pod "fb78abf6-e9ea-49fa-ac96-87363bfd5974" (UID: "fb78abf6-e9ea-49fa-ac96-87363bfd5974"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:30:36.892888 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.892851    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb78abf6-e9ea-49fa-ac96-87363bfd5974-console-config" (OuterVolumeSpecName: "console-config") pod "fb78abf6-e9ea-49fa-ac96-87363bfd5974" (UID: "fb78abf6-e9ea-49fa-ac96-87363bfd5974"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:30:36.892972 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.892904    2574 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fb78abf6-e9ea-49fa-ac96-87363bfd5974-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:30:36.893284 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.893176    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb78abf6-e9ea-49fa-ac96-87363bfd5974-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fb78abf6-e9ea-49fa-ac96-87363bfd5974" (UID: "fb78abf6-e9ea-49fa-ac96-87363bfd5974"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:30:36.893284 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.893288    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fb78abf6-e9ea-49fa-ac96-87363bfd5974-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fb78abf6-e9ea-49fa-ac96-87363bfd5974" (UID: "fb78abf6-e9ea-49fa-ac96-87363bfd5974"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:30:36.894847 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.894801    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb78abf6-e9ea-49fa-ac96-87363bfd5974-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fb78abf6-e9ea-49fa-ac96-87363bfd5974" (UID: "fb78abf6-e9ea-49fa-ac96-87363bfd5974"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:30:36.894960 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.894883    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb78abf6-e9ea-49fa-ac96-87363bfd5974-kube-api-access-lqcbz" (OuterVolumeSpecName: "kube-api-access-lqcbz") pod "fb78abf6-e9ea-49fa-ac96-87363bfd5974" (UID: "fb78abf6-e9ea-49fa-ac96-87363bfd5974"). InnerVolumeSpecName "kube-api-access-lqcbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:30:36.895341 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.895309    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fb78abf6-e9ea-49fa-ac96-87363bfd5974-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fb78abf6-e9ea-49fa-ac96-87363bfd5974" (UID: "fb78abf6-e9ea-49fa-ac96-87363bfd5974"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:30:36.896809 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.896787    2574 scope.go:117] "RemoveContainer" containerID="873bfa7b7f86e1e798707964cf77e5d357416aee6ae00236c25ec8803c2eab55"
Jul 17 09:30:36.897120 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:30:36.897097    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"873bfa7b7f86e1e798707964cf77e5d357416aee6ae00236c25ec8803c2eab55\": container with ID starting with 873bfa7b7f86e1e798707964cf77e5d357416aee6ae00236c25ec8803c2eab55 not found: ID does not exist" containerID="873bfa7b7f86e1e798707964cf77e5d357416aee6ae00236c25ec8803c2eab55"
Jul 17 09:30:36.897177 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.897132    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"873bfa7b7f86e1e798707964cf77e5d357416aee6ae00236c25ec8803c2eab55"} err="failed to get container status \"873bfa7b7f86e1e798707964cf77e5d357416aee6ae00236c25ec8803c2eab55\": rpc error: code = NotFound desc = could not find container \"873bfa7b7f86e1e798707964cf77e5d357416aee6ae00236c25ec8803c2eab55\": container with ID starting with 873bfa7b7f86e1e798707964cf77e5d357416aee6ae00236c25ec8803c2eab55 not found: ID does not exist"
Jul 17 09:30:36.993763 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.993674    2574 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fb78abf6-e9ea-49fa-ac96-87363bfd5974-console-config\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:30:36.993763 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.993703    2574 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fb78abf6-e9ea-49fa-ac96-87363bfd5974-console-oauth-config\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:30:36.993763 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.993714    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqcbz\" (UniqueName: \"kubernetes.io/projected/fb78abf6-e9ea-49fa-ac96-87363bfd5974-kube-api-access-lqcbz\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:30:36.993763 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.993723    2574 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fb78abf6-e9ea-49fa-ac96-87363bfd5974-oauth-serving-cert\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:30:36.993763 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.993731    2574 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fb78abf6-e9ea-49fa-ac96-87363bfd5974-trusted-ca-bundle\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:30:36.993763 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:36.993739    2574 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fb78abf6-e9ea-49fa-ac96-87363bfd5974-console-serving-cert\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:30:37.202852 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:37.202815    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-fdc6688f4-7rzg8"]
Jul 17 09:30:37.206940 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:37.206909    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-fdc6688f4-7rzg8"]
Jul 17 09:30:38.895923 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:30:38.895886    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb78abf6-e9ea-49fa-ac96-87363bfd5974" path="/var/lib/kubelet/pods/fb78abf6-e9ea-49fa-ac96-87363bfd5974/volumes"
Jul 17 09:31:23.848030 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:23.847993    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tr7rr"]
Jul 17 09:31:23.848493 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:23.848293    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a83a14c-2f3b-4482-8557-131da3e63529" containerName="console"
Jul 17 09:31:23.848493 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:23.848304    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a83a14c-2f3b-4482-8557-131da3e63529" containerName="console"
Jul 17 09:31:23.848493 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:23.848314    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb78abf6-e9ea-49fa-ac96-87363bfd5974" containerName="console"
Jul 17 09:31:23.848493 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:23.848319    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb78abf6-e9ea-49fa-ac96-87363bfd5974" containerName="console"
Jul 17 09:31:23.848493 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:23.848335    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b1d64eb-7388-44f0-8600-2dcbea6896c7" containerName="console"
Jul 17 09:31:23.848493 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:23.848340    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b1d64eb-7388-44f0-8600-2dcbea6896c7" containerName="console"
Jul 17 09:31:23.848493 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:23.848350    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27c6d907-d449-4202-8700-dc369dadb9f4" containerName="registry"
Jul 17 09:31:23.848493 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:23.848355    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="27c6d907-d449-4202-8700-dc369dadb9f4" containerName="registry"
Jul 17 09:31:23.848493 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:23.848400    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb78abf6-e9ea-49fa-ac96-87363bfd5974" containerName="console"
Jul 17 09:31:23.848493 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:23.848409    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="27c6d907-d449-4202-8700-dc369dadb9f4" containerName="registry"
Jul 17 09:31:23.848493 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:23.848416    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a83a14c-2f3b-4482-8557-131da3e63529" containerName="console"
Jul 17 09:31:23.848493 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:23.848422    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b1d64eb-7388-44f0-8600-2dcbea6896c7" containerName="console"
Jul 17 09:31:23.850456 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:23.850441    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-tr7rr"
Jul 17 09:31:23.853163 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:23.853141    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 17 09:31:23.853316 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:23.853174    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 17 09:31:23.854499 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:23.854475    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-k6nbw\""
Jul 17 09:31:23.859429 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:23.859409    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tr7rr"]
Jul 17 09:31:23.974590 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:23.974550    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90f4e5b3-c039-472f-b206-d6f1600af3b1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tr7rr\" (UID: \"90f4e5b3-c039-472f-b206-d6f1600af3b1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tr7rr"
Jul 17 09:31:23.974762 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:23.974621    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2nvr\" (UniqueName: \"kubernetes.io/projected/90f4e5b3-c039-472f-b206-d6f1600af3b1-kube-api-access-f2nvr\") pod \"cert-manager-webhook-6486ffd997-tr7rr\" (UID: \"90f4e5b3-c039-472f-b206-d6f1600af3b1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tr7rr"
Jul 17 09:31:24.075586 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:24.075552    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90f4e5b3-c039-472f-b206-d6f1600af3b1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tr7rr\" (UID: \"90f4e5b3-c039-472f-b206-d6f1600af3b1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tr7rr"
Jul 17 09:31:24.075746 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:24.075618    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2nvr\" (UniqueName: \"kubernetes.io/projected/90f4e5b3-c039-472f-b206-d6f1600af3b1-kube-api-access-f2nvr\") pod \"cert-manager-webhook-6486ffd997-tr7rr\" (UID: \"90f4e5b3-c039-472f-b206-d6f1600af3b1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tr7rr"
Jul 17 09:31:24.084094 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:24.084061    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/90f4e5b3-c039-472f-b206-d6f1600af3b1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-tr7rr\" (UID: \"90f4e5b3-c039-472f-b206-d6f1600af3b1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tr7rr"
Jul 17 09:31:24.084198 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:24.084096    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2nvr\" (UniqueName: \"kubernetes.io/projected/90f4e5b3-c039-472f-b206-d6f1600af3b1-kube-api-access-f2nvr\") pod \"cert-manager-webhook-6486ffd997-tr7rr\" (UID: \"90f4e5b3-c039-472f-b206-d6f1600af3b1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-tr7rr"
Jul 17 09:31:24.160346 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:24.160322    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-tr7rr"
Jul 17 09:31:24.286196 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:24.286174    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-tr7rr"]
Jul 17 09:31:24.288870 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:31:24.288843    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod90f4e5b3_c039_472f_b206_d6f1600af3b1.slice/crio-5bc73da1a955b6ff55185d8012ee0b817e03feea28ff8c91b71e9f318ec4f4e2 WatchSource:0}: Error finding container 5bc73da1a955b6ff55185d8012ee0b817e03feea28ff8c91b71e9f318ec4f4e2: Status 404 returned error can't find the container with id 5bc73da1a955b6ff55185d8012ee0b817e03feea28ff8c91b71e9f318ec4f4e2
Jul 17 09:31:25.072588 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:25.072548    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-tr7rr" event={"ID":"90f4e5b3-c039-472f-b206-d6f1600af3b1","Type":"ContainerStarted","Data":"5bc73da1a955b6ff55185d8012ee0b817e03feea28ff8c91b71e9f318ec4f4e2"}
Jul 17 09:31:28.087053 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:28.087015    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-tr7rr" event={"ID":"90f4e5b3-c039-472f-b206-d6f1600af3b1","Type":"ContainerStarted","Data":"369e171d29e54cc7756fd4cdc0df0edaa076dabfc6463319403a1fcd9b5620e4"}
Jul 17 09:31:28.087467 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:28.087087    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-tr7rr"
Jul 17 09:31:28.105304 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:28.105231    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-tr7rr" podStartSLOduration=1.958701569 podStartE2EDuration="5.10521716s" totalImagesPullingTime="3.146515591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:31:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:31:28.103496678 +0000 UTC m=+285.686135264" watchObservedRunningTime="2026-07-17 09:31:28.10521716 +0000 UTC m=+285.687855535"
Jul 17 09:31:30.816008 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:30.815969    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nzndn"]
Jul 17 09:31:30.819139 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:30.819123    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nzndn"
Jul 17 09:31:30.821938 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:30.821920    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 17 09:31:30.822015 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:30.821950    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 17 09:31:30.823279 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:30.823266    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-vdrtg\""
Jul 17 09:31:30.828688 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:30.828664    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nzndn"]
Jul 17 09:31:30.936230 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:30.936192    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwcjc\" (UniqueName: \"kubernetes.io/projected/765b8fd3-de37-4e60-84ea-54f2007bbf22-kube-api-access-zwcjc\") pod \"openshift-lws-operator-fd8ccff4c-nzndn\" (UID: \"765b8fd3-de37-4e60-84ea-54f2007bbf22\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nzndn"
Jul 17 09:31:30.936448 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:30.936314    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/765b8fd3-de37-4e60-84ea-54f2007bbf22-tmp\") pod \"openshift-lws-operator-fd8ccff4c-nzndn\" (UID: \"765b8fd3-de37-4e60-84ea-54f2007bbf22\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nzndn"
Jul 17 09:31:31.037206 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:31.037171    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwcjc\" (UniqueName: \"kubernetes.io/projected/765b8fd3-de37-4e60-84ea-54f2007bbf22-kube-api-access-zwcjc\") pod \"openshift-lws-operator-fd8ccff4c-nzndn\" (UID: \"765b8fd3-de37-4e60-84ea-54f2007bbf22\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nzndn"
Jul 17 09:31:31.037424 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:31.037267    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/765b8fd3-de37-4e60-84ea-54f2007bbf22-tmp\") pod \"openshift-lws-operator-fd8ccff4c-nzndn\" (UID: \"765b8fd3-de37-4e60-84ea-54f2007bbf22\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nzndn"
Jul 17 09:31:31.037697 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:31.037672    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/765b8fd3-de37-4e60-84ea-54f2007bbf22-tmp\") pod \"openshift-lws-operator-fd8ccff4c-nzndn\" (UID: \"765b8fd3-de37-4e60-84ea-54f2007bbf22\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nzndn"
Jul 17 09:31:31.045820 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:31.045795    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwcjc\" (UniqueName: \"kubernetes.io/projected/765b8fd3-de37-4e60-84ea-54f2007bbf22-kube-api-access-zwcjc\") pod \"openshift-lws-operator-fd8ccff4c-nzndn\" (UID: \"765b8fd3-de37-4e60-84ea-54f2007bbf22\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nzndn"
Jul 17 09:31:31.128510 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:31.128415    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nzndn"
Jul 17 09:31:31.257459 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:31.257430    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nzndn"]
Jul 17 09:31:31.260054 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:31:31.260025    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod765b8fd3_de37_4e60_84ea_54f2007bbf22.slice/crio-fa661b347522b497ca165d07f52544042bb075b9bcc841fe80d5fd197ab4494c WatchSource:0}: Error finding container fa661b347522b497ca165d07f52544042bb075b9bcc841fe80d5fd197ab4494c: Status 404 returned error can't find the container with id fa661b347522b497ca165d07f52544042bb075b9bcc841fe80d5fd197ab4494c
Jul 17 09:31:32.104973 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:32.104931    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nzndn" event={"ID":"765b8fd3-de37-4e60-84ea-54f2007bbf22","Type":"ContainerStarted","Data":"fa661b347522b497ca165d07f52544042bb075b9bcc841fe80d5fd197ab4494c"}
Jul 17 09:31:34.093678 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:34.093592    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-tr7rr"
Jul 17 09:31:34.114921 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:34.114884    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nzndn" event={"ID":"765b8fd3-de37-4e60-84ea-54f2007bbf22","Type":"ContainerStarted","Data":"bf2ee4f61b88ec65ce0a6eeb5a3cd48e4ee3f0df902c1a9e46c35a12fd531742"}
Jul 17 09:31:34.131764 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:34.131717    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-nzndn" podStartSLOduration=1.580311426 podStartE2EDuration="4.131701947s" totalImagesPullingTime="2.551390521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:31:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:31:34.131025056 +0000 UTC m=+291.713663430" watchObservedRunningTime="2026-07-17 09:31:34.131701947 +0000 UTC m=+291.714340322"
Jul 17 09:31:42.803110 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:42.803077    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dh564_84db06ab-87e2-43b3-82d0-13f06ba86d06/ovn-acl-logging/0.log"
Jul 17 09:31:42.807759 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:42.807732    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dh564_84db06ab-87e2-43b3-82d0-13f06ba86d06/ovn-acl-logging/0.log"
Jul 17 09:31:42.809945 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:42.809923    2574 kubelet.go:1648] "Image garbage collection succeeded"
Jul 17 09:31:51.495103 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:51.495066    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-ndp9h"]
Jul 17 09:31:51.557485 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:51.557457    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-ndp9h"]
Jul 17 09:31:51.557611 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:51.557582    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-ndp9h"
Jul 17 09:31:51.561337 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:51.561312    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7g64w\""
Jul 17 09:31:51.561460 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:51.561348    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 17 09:31:51.561611 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:51.561589    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 17 09:31:51.561836 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:51.561822    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 17 09:31:51.561874 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:51.561838    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 17 09:31:51.710169 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:51.710128    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce30ed03-36b2-442a-8dba-13d434d42d39-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-ndp9h\" (UID: \"ce30ed03-36b2-442a-8dba-13d434d42d39\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-ndp9h"
Jul 17 09:31:51.710169 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:51.710176    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ce30ed03-36b2-442a-8dba-13d434d42d39-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-ndp9h\" (UID: \"ce30ed03-36b2-442a-8dba-13d434d42d39\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-ndp9h"
Jul 17 09:31:51.710474 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:51.710199    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v5vv\" (UniqueName: \"kubernetes.io/projected/ce30ed03-36b2-442a-8dba-13d434d42d39-kube-api-access-4v5vv\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-ndp9h\" (UID: \"ce30ed03-36b2-442a-8dba-13d434d42d39\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-ndp9h"
Jul 17 09:31:51.710474 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:51.710342    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce30ed03-36b2-442a-8dba-13d434d42d39-tmp\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-ndp9h\" (UID: \"ce30ed03-36b2-442a-8dba-13d434d42d39\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-ndp9h"
Jul 17 09:31:51.710474 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:51.710393    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ce30ed03-36b2-442a-8dba-13d434d42d39-manifests\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-ndp9h\" (UID: \"ce30ed03-36b2-442a-8dba-13d434d42d39\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-ndp9h"
Jul 17 09:31:51.811224 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:51.811140    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ce30ed03-36b2-442a-8dba-13d434d42d39-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-ndp9h\" (UID: \"ce30ed03-36b2-442a-8dba-13d434d42d39\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-ndp9h"
Jul 17 09:31:51.811224 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:51.811175    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v5vv\" (UniqueName: \"kubernetes.io/projected/ce30ed03-36b2-442a-8dba-13d434d42d39-kube-api-access-4v5vv\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-ndp9h\" (UID: \"ce30ed03-36b2-442a-8dba-13d434d42d39\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-ndp9h"
Jul 17 09:31:51.811453 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:51.811272    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce30ed03-36b2-442a-8dba-13d434d42d39-tmp\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-ndp9h\" (UID: \"ce30ed03-36b2-442a-8dba-13d434d42d39\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-ndp9h"
Jul 17 09:31:51.811453 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:51.811300    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ce30ed03-36b2-442a-8dba-13d434d42d39-manifests\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-ndp9h\" (UID: \"ce30ed03-36b2-442a-8dba-13d434d42d39\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-ndp9h"
Jul 17 09:31:51.811453 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:51.811346    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce30ed03-36b2-442a-8dba-13d434d42d39-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-ndp9h\" (UID: \"ce30ed03-36b2-442a-8dba-13d434d42d39\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-ndp9h"
Jul 17 09:31:51.811768 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:51.811744    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ce30ed03-36b2-442a-8dba-13d434d42d39-manifests\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-ndp9h\" (UID: \"ce30ed03-36b2-442a-8dba-13d434d42d39\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-ndp9h"
Jul 17 09:31:51.813645 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:51.813619    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce30ed03-36b2-442a-8dba-13d434d42d39-tmp\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-ndp9h\" (UID: \"ce30ed03-36b2-442a-8dba-13d434d42d39\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-ndp9h"
Jul 17 09:31:51.813747 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:51.813700    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ce30ed03-36b2-442a-8dba-13d434d42d39-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-ndp9h\" (UID: \"ce30ed03-36b2-442a-8dba-13d434d42d39\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-ndp9h"
Jul 17 09:31:51.813830 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:51.813812    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce30ed03-36b2-442a-8dba-13d434d42d39-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-ndp9h\" (UID: \"ce30ed03-36b2-442a-8dba-13d434d42d39\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-ndp9h"
Jul 17 09:31:51.820773 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:51.820743    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v5vv\" (UniqueName: \"kubernetes.io/projected/ce30ed03-36b2-442a-8dba-13d434d42d39-kube-api-access-4v5vv\") pod \"opendatahub-operator-controller-manager-6b5566fdf9-ndp9h\" (UID: \"ce30ed03-36b2-442a-8dba-13d434d42d39\") " pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-ndp9h"
Jul 17 09:31:51.867499 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:51.867459    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-ndp9h"
Jul 17 09:31:52.023281 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:52.023231    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-ndp9h"]
Jul 17 09:31:52.026803 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:31:52.026767    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce30ed03_36b2_442a_8dba_13d434d42d39.slice/crio-1fc7aa8eab214fa9b51f80cef87020592fe09844b833723bce7e091323e46e1d WatchSource:0}: Error finding container 1fc7aa8eab214fa9b51f80cef87020592fe09844b833723bce7e091323e46e1d: Status 404 returned error can't find the container with id 1fc7aa8eab214fa9b51f80cef87020592fe09844b833723bce7e091323e46e1d
Jul 17 09:31:52.028685 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:52.028657    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 09:31:52.188858 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:52.188822    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-ndp9h" event={"ID":"ce30ed03-36b2-442a-8dba-13d434d42d39","Type":"ContainerStarted","Data":"1fc7aa8eab214fa9b51f80cef87020592fe09844b833723bce7e091323e46e1d"}
Jul 17 09:31:55.212560 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:55.212514    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-ndp9h" event={"ID":"ce30ed03-36b2-442a-8dba-13d434d42d39","Type":"ContainerStarted","Data":"6a9b61dab8c2ba424bd68b8350e611344ea494c33cf5a00a77c03968a972872a"}
Jul 17 09:31:56.218251 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:56.218214    2574 generic.go:356] "Generic (PLEG): container finished" podID="ce30ed03-36b2-442a-8dba-13d434d42d39" containerID="6a9b61dab8c2ba424bd68b8350e611344ea494c33cf5a00a77c03968a972872a" exitCode=0
Jul 17 09:31:56.218632 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:56.218300    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-ndp9h" event={"ID":"ce30ed03-36b2-442a-8dba-13d434d42d39","Type":"ContainerDied","Data":"6a9b61dab8c2ba424bd68b8350e611344ea494c33cf5a00a77c03968a972872a"}
Jul 17 09:31:57.225456 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:57.225416    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-ndp9h" event={"ID":"ce30ed03-36b2-442a-8dba-13d434d42d39","Type":"ContainerStarted","Data":"be1128c4ff1acd8f551cd5a45807afd92dbe141835a51bfba1c4e657ce1deeae"}
Jul 17 09:31:57.225879 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:57.225633    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-ndp9h"
Jul 17 09:31:57.248654 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:31:57.248601    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-ndp9h" podStartSLOduration=1.857496175 podStartE2EDuration="6.248584179s" totalImagesPullingTime="3.337174289s" totalInitContainerRuntime="1.053913715s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:31:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:31:57.248126675 +0000 UTC m=+314.830765055" watchObservedRunningTime="2026-07-17 09:31:57.248584179 +0000 UTC m=+314.831222555"
Jul 17 09:32:08.232071 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:08.232038    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6b5566fdf9-ndp9h"
Jul 17 09:32:11.941620 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:11.941585    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-546bc49978-vw2hd"]
Jul 17 09:32:11.944002 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:11.943970    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-546bc49978-vw2hd"
Jul 17 09:32:11.946792 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:11.946768    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 17 09:32:11.947134 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:11.947114    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 17 09:32:11.947193 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:11.947128    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 17 09:32:11.948084 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:11.948056    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 17 09:32:11.959605 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:11.959586    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-546bc49978-vw2hd"]
Jul 17 09:32:11.978771 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:11.978749    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f7ee6c1-0dfe-4612-93a9-beccaa966946-tls-certs\") pod \"kube-auth-proxy-546bc49978-vw2hd\" (UID: \"1f7ee6c1-0dfe-4612-93a9-beccaa966946\") " pod="openshift-ingress/kube-auth-proxy-546bc49978-vw2hd"
Jul 17 09:32:11.978871 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:11.978835    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f7ee6c1-0dfe-4612-93a9-beccaa966946-tmp\") pod \"kube-auth-proxy-546bc49978-vw2hd\" (UID: \"1f7ee6c1-0dfe-4612-93a9-beccaa966946\") " pod="openshift-ingress/kube-auth-proxy-546bc49978-vw2hd"
Jul 17 09:32:11.978937 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:11.978883    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbhqp\" (UniqueName: \"kubernetes.io/projected/1f7ee6c1-0dfe-4612-93a9-beccaa966946-kube-api-access-mbhqp\") pod \"kube-auth-proxy-546bc49978-vw2hd\" (UID: \"1f7ee6c1-0dfe-4612-93a9-beccaa966946\") " pod="openshift-ingress/kube-auth-proxy-546bc49978-vw2hd"
Jul 17 09:32:12.080124 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:12.080099    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f7ee6c1-0dfe-4612-93a9-beccaa966946-tmp\") pod \"kube-auth-proxy-546bc49978-vw2hd\" (UID: \"1f7ee6c1-0dfe-4612-93a9-beccaa966946\") " pod="openshift-ingress/kube-auth-proxy-546bc49978-vw2hd"
Jul 17 09:32:12.080291 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:12.080140    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbhqp\" (UniqueName: \"kubernetes.io/projected/1f7ee6c1-0dfe-4612-93a9-beccaa966946-kube-api-access-mbhqp\") pod \"kube-auth-proxy-546bc49978-vw2hd\" (UID: \"1f7ee6c1-0dfe-4612-93a9-beccaa966946\") " pod="openshift-ingress/kube-auth-proxy-546bc49978-vw2hd"
Jul 17 09:32:12.080291 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:12.080166    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f7ee6c1-0dfe-4612-93a9-beccaa966946-tls-certs\") pod \"kube-auth-proxy-546bc49978-vw2hd\" (UID: \"1f7ee6c1-0dfe-4612-93a9-beccaa966946\") " pod="openshift-ingress/kube-auth-proxy-546bc49978-vw2hd"
Jul 17 09:32:12.082340 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:12.082312    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f7ee6c1-0dfe-4612-93a9-beccaa966946-tmp\") pod \"kube-auth-proxy-546bc49978-vw2hd\" (UID: \"1f7ee6c1-0dfe-4612-93a9-beccaa966946\") " pod="openshift-ingress/kube-auth-proxy-546bc49978-vw2hd"
Jul 17 09:32:12.082580 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:12.082562    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f7ee6c1-0dfe-4612-93a9-beccaa966946-tls-certs\") pod \"kube-auth-proxy-546bc49978-vw2hd\" (UID: \"1f7ee6c1-0dfe-4612-93a9-beccaa966946\") " pod="openshift-ingress/kube-auth-proxy-546bc49978-vw2hd"
Jul 17 09:32:12.101199 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:12.101159    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbhqp\" (UniqueName: \"kubernetes.io/projected/1f7ee6c1-0dfe-4612-93a9-beccaa966946-kube-api-access-mbhqp\") pod \"kube-auth-proxy-546bc49978-vw2hd\" (UID: \"1f7ee6c1-0dfe-4612-93a9-beccaa966946\") " pod="openshift-ingress/kube-auth-proxy-546bc49978-vw2hd"
Jul 17 09:32:12.150263 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:12.150215    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bs5g8"]
Jul 17 09:32:12.152793 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:12.152777    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-bs5g8"
Jul 17 09:32:12.156583 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:12.156560    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 17 09:32:12.157143 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:12.157128    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-7w7t7\""
Jul 17 09:32:12.172717 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:12.172697    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bs5g8"]
Jul 17 09:32:12.181100 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:12.181074    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2cdf35a8-c7a2-49c1-98d5-7153f317c517-cert\") pod \"odh-model-controller-7998c67dc4-bs5g8\" (UID: \"2cdf35a8-c7a2-49c1-98d5-7153f317c517\") " pod="opendatahub/odh-model-controller-7998c67dc4-bs5g8"
Jul 17 09:32:12.181188 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:12.181143    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nts47\" (UniqueName: \"kubernetes.io/projected/2cdf35a8-c7a2-49c1-98d5-7153f317c517-kube-api-access-nts47\") pod \"odh-model-controller-7998c67dc4-bs5g8\" (UID: \"2cdf35a8-c7a2-49c1-98d5-7153f317c517\") " pod="opendatahub/odh-model-controller-7998c67dc4-bs5g8"
Jul 17 09:32:12.253732 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:12.253647    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-546bc49978-vw2hd"
Jul 17 09:32:12.282335 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:12.282310    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2cdf35a8-c7a2-49c1-98d5-7153f317c517-cert\") pod \"odh-model-controller-7998c67dc4-bs5g8\" (UID: \"2cdf35a8-c7a2-49c1-98d5-7153f317c517\") " pod="opendatahub/odh-model-controller-7998c67dc4-bs5g8"
Jul 17 09:32:12.282462 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:12.282359    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nts47\" (UniqueName: \"kubernetes.io/projected/2cdf35a8-c7a2-49c1-98d5-7153f317c517-kube-api-access-nts47\") pod \"odh-model-controller-7998c67dc4-bs5g8\" (UID: \"2cdf35a8-c7a2-49c1-98d5-7153f317c517\") " pod="opendatahub/odh-model-controller-7998c67dc4-bs5g8"
Jul 17 09:32:12.284871 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:12.284844    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2cdf35a8-c7a2-49c1-98d5-7153f317c517-cert\") pod \"odh-model-controller-7998c67dc4-bs5g8\" (UID: \"2cdf35a8-c7a2-49c1-98d5-7153f317c517\") " pod="opendatahub/odh-model-controller-7998c67dc4-bs5g8"
Jul 17 09:32:12.296630 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:12.296604    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nts47\" (UniqueName: \"kubernetes.io/projected/2cdf35a8-c7a2-49c1-98d5-7153f317c517-kube-api-access-nts47\") pod \"odh-model-controller-7998c67dc4-bs5g8\" (UID: \"2cdf35a8-c7a2-49c1-98d5-7153f317c517\") " pod="opendatahub/odh-model-controller-7998c67dc4-bs5g8"
Jul 17 09:32:12.400273 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:12.400229    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-546bc49978-vw2hd"]
Jul 17 09:32:12.402494 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:32:12.402445    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f7ee6c1_0dfe_4612_93a9_beccaa966946.slice/crio-e7c6bd5af19b700eec79246cb4c8a31b88b9022632ac56ca123ee4c3760332e0 WatchSource:0}: Error finding container e7c6bd5af19b700eec79246cb4c8a31b88b9022632ac56ca123ee4c3760332e0: Status 404 returned error can't find the container with id e7c6bd5af19b700eec79246cb4c8a31b88b9022632ac56ca123ee4c3760332e0
Jul 17 09:32:12.463584 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:12.463553    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-bs5g8"
Jul 17 09:32:12.598232 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:12.598209    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bs5g8"]
Jul 17 09:32:12.600850 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:32:12.600825    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cdf35a8_c7a2_49c1_98d5_7153f317c517.slice/crio-d3b62777b4be29047e3885c970db5ef0141cf0fc39e64ff0f53bcf0febd853e6 WatchSource:0}: Error finding container d3b62777b4be29047e3885c970db5ef0141cf0fc39e64ff0f53bcf0febd853e6: Status 404 returned error can't find the container with id d3b62777b4be29047e3885c970db5ef0141cf0fc39e64ff0f53bcf0febd853e6
Jul 17 09:32:13.053550 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.053514    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-96757"]
Jul 17 09:32:13.056824 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.056799    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-96757"
Jul 17 09:32:13.059574 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.059548    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 17 09:32:13.066912 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.066885    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-96757"]
Jul 17 09:32:13.088197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.087998    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e56c1862-83dd-417d-bd30-60abc092e276-tls-certs\") pod \"model-serving-api-54d89bd5fc-96757\" (UID: \"e56c1862-83dd-417d-bd30-60abc092e276\") " pod="opendatahub/model-serving-api-54d89bd5fc-96757"
Jul 17 09:32:13.088197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.088059    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gvgs\" (UniqueName: \"kubernetes.io/projected/e56c1862-83dd-417d-bd30-60abc092e276-kube-api-access-9gvgs\") pod \"model-serving-api-54d89bd5fc-96757\" (UID: \"e56c1862-83dd-417d-bd30-60abc092e276\") " pod="opendatahub/model-serving-api-54d89bd5fc-96757"
Jul 17 09:32:13.189445 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.189412    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e56c1862-83dd-417d-bd30-60abc092e276-tls-certs\") pod \"model-serving-api-54d89bd5fc-96757\" (UID: \"e56c1862-83dd-417d-bd30-60abc092e276\") " pod="opendatahub/model-serving-api-54d89bd5fc-96757"
Jul 17 09:32:13.189590 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.189461    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gvgs\" (UniqueName: \"kubernetes.io/projected/e56c1862-83dd-417d-bd30-60abc092e276-kube-api-access-9gvgs\") pod \"model-serving-api-54d89bd5fc-96757\" (UID: \"e56c1862-83dd-417d-bd30-60abc092e276\") " pod="opendatahub/model-serving-api-54d89bd5fc-96757"
Jul 17 09:32:13.189590 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:32:13.189572    2574 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 17 09:32:13.189672 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:32:13.189647    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e56c1862-83dd-417d-bd30-60abc092e276-tls-certs podName:e56c1862-83dd-417d-bd30-60abc092e276 nodeName:}" failed. No retries permitted until 2026-07-17 09:32:13.689626418 +0000 UTC m=+331.272264771 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/e56c1862-83dd-417d-bd30-60abc092e276-tls-certs") pod "model-serving-api-54d89bd5fc-96757" (UID: "e56c1862-83dd-417d-bd30-60abc092e276") : secret "model-serving-api-tls" not found
Jul 17 09:32:13.200627 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.200598    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gvgs\" (UniqueName: \"kubernetes.io/projected/e56c1862-83dd-417d-bd30-60abc092e276-kube-api-access-9gvgs\") pod \"model-serving-api-54d89bd5fc-96757\" (UID: \"e56c1862-83dd-417d-bd30-60abc092e276\") " pod="opendatahub/model-serving-api-54d89bd5fc-96757"
Jul 17 09:32:13.269912 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.269878    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"]
Jul 17 09:32:13.272865 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.272837    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"
Jul 17 09:32:13.275447 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.275422    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 17 09:32:13.275644 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.275627    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 17 09:32:13.275812 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.275791    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 17 09:32:13.275953 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.275927    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-pxfmw\""
Jul 17 09:32:13.276027 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.276014    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 17 09:32:13.289988 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.289930    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-qvd4s\" (UID: \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"
Jul 17 09:32:13.290096 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.290005    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-qvd4s\" (UID: \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"
Jul 17 09:32:13.290096 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.290035    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-cacerts\") pod \"istiod-openshift-gateway-7574459656-qvd4s\" (UID: \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"
Jul 17 09:32:13.290096 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.290057    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-istio-token\") pod \"istiod-openshift-gateway-7574459656-qvd4s\" (UID: \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"
Jul 17 09:32:13.290096 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.290057    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"]
Jul 17 09:32:13.290096 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.290090    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-qvd4s\" (UID: \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"
Jul 17 09:32:13.290334 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.290175    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9rzf\" (UniqueName: \"kubernetes.io/projected/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-kube-api-access-c9rzf\") pod \"istiod-openshift-gateway-7574459656-qvd4s\" (UID: \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"
Jul 17 09:32:13.290334 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.290201    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-local-certs\") pod \"istiod-openshift-gateway-7574459656-qvd4s\" (UID: \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"
Jul 17 09:32:13.297589 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.297511    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-bs5g8" event={"ID":"2cdf35a8-c7a2-49c1-98d5-7153f317c517","Type":"ContainerStarted","Data":"d3b62777b4be29047e3885c970db5ef0141cf0fc39e64ff0f53bcf0febd853e6"}
Jul 17 09:32:13.299832 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.299787    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-546bc49978-vw2hd" event={"ID":"1f7ee6c1-0dfe-4612-93a9-beccaa966946","Type":"ContainerStarted","Data":"e7c6bd5af19b700eec79246cb4c8a31b88b9022632ac56ca123ee4c3760332e0"}
Jul 17 09:32:13.391718 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.391620    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-qvd4s\" (UID: \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"
Jul 17 09:32:13.391718 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.391664    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-cacerts\") pod \"istiod-openshift-gateway-7574459656-qvd4s\" (UID: \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"
Jul 17 09:32:13.391718 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.391694    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-istio-token\") pod \"istiod-openshift-gateway-7574459656-qvd4s\" (UID: \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"
Jul 17 09:32:13.391972 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.391724    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-qvd4s\" (UID: \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"
Jul 17 09:32:13.391972 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.391774    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9rzf\" (UniqueName: \"kubernetes.io/projected/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-kube-api-access-c9rzf\") pod \"istiod-openshift-gateway-7574459656-qvd4s\" (UID: \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"
Jul 17 09:32:13.391972 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.391800    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-local-certs\") pod \"istiod-openshift-gateway-7574459656-qvd4s\" (UID: \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"
Jul 17 09:32:13.391972 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.391870    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-qvd4s\" (UID: \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"
Jul 17 09:32:13.393278 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.393210    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-qvd4s\" (UID: \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"
Jul 17 09:32:13.395686 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.395595    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-local-certs\") pod \"istiod-openshift-gateway-7574459656-qvd4s\" (UID: \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"
Jul 17 09:32:13.395916 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.395896    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-qvd4s\" (UID: \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"
Jul 17 09:32:13.396840 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.396784    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-qvd4s\" (UID: \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"
Jul 17 09:32:13.397465 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.397441    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-cacerts\") pod \"istiod-openshift-gateway-7574459656-qvd4s\" (UID: \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"
Jul 17 09:32:13.402611 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.402548    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-istio-token\") pod \"istiod-openshift-gateway-7574459656-qvd4s\" (UID: \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"
Jul 17 09:32:13.402923 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.402890    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9rzf\" (UniqueName: \"kubernetes.io/projected/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-kube-api-access-c9rzf\") pod \"istiod-openshift-gateway-7574459656-qvd4s\" (UID: \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"
Jul 17 09:32:13.584590 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.584344    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"
Jul 17 09:32:13.695089 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.694689    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e56c1862-83dd-417d-bd30-60abc092e276-tls-certs\") pod \"model-serving-api-54d89bd5fc-96757\" (UID: \"e56c1862-83dd-417d-bd30-60abc092e276\") " pod="opendatahub/model-serving-api-54d89bd5fc-96757"
Jul 17 09:32:13.698593 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.698544    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e56c1862-83dd-417d-bd30-60abc092e276-tls-certs\") pod \"model-serving-api-54d89bd5fc-96757\" (UID: \"e56c1862-83dd-417d-bd30-60abc092e276\") " pod="opendatahub/model-serving-api-54d89bd5fc-96757"
Jul 17 09:32:13.778128 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.778101    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"]
Jul 17 09:32:13.781347 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:32:13.781318    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6add91c8_e0a7_4f12_9ce6_669a8bbbc160.slice/crio-e67777653be4cff760a7bb5fd076964d1f61395344c5145fe370f339462f2ea3 WatchSource:0}: Error finding container e67777653be4cff760a7bb5fd076964d1f61395344c5145fe370f339462f2ea3: Status 404 returned error can't find the container with id e67777653be4cff760a7bb5fd076964d1f61395344c5145fe370f339462f2ea3
Jul 17 09:32:13.972346 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:13.972232    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-96757"
Jul 17 09:32:14.147326 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:14.147216    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-96757"]
Jul 17 09:32:14.309689 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:14.309598    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s" event={"ID":"6add91c8-e0a7-4f12-9ce6-669a8bbbc160","Type":"ContainerStarted","Data":"e67777653be4cff760a7bb5fd076964d1f61395344c5145fe370f339462f2ea3"}
Jul 17 09:32:14.311314 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:14.311284    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-96757" event={"ID":"e56c1862-83dd-417d-bd30-60abc092e276","Type":"ContainerStarted","Data":"34c613502c6fda52b758765a3ef268106aee61c702cce1b1b51e330adac61b1f"}
Jul 17 09:32:17.330971 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:17.330900    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-bs5g8" event={"ID":"2cdf35a8-c7a2-49c1-98d5-7153f317c517","Type":"ContainerStarted","Data":"de3eca23b06f206c8f40bb28836c3a69f440245a52ea24a49e7620a21f4c94bd"}
Jul 17 09:32:17.331481 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:17.331020    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-bs5g8"
Jul 17 09:32:17.332714 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:17.332671    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-546bc49978-vw2hd" event={"ID":"1f7ee6c1-0dfe-4612-93a9-beccaa966946","Type":"ContainerStarted","Data":"e3f9f43136e62db1f2b6b719f87979090985999a7cf58261ed9da47442e80ceb"}
Jul 17 09:32:17.346698 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:17.346647    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-bs5g8" podStartSLOduration=1.523606577 podStartE2EDuration="5.346632615s" totalImagesPullingTime="3.823026038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:32:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:32:17.345556332 +0000 UTC m=+334.928194707" watchObservedRunningTime="2026-07-17 09:32:17.346632615 +0000 UTC m=+334.929270990"
Jul 17 09:32:17.360687 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:17.360389    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-546bc49978-vw2hd" podStartSLOduration=2.353410851 podStartE2EDuration="6.360373849s" totalImagesPullingTime="4.006962998s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:32:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:32:17.360371124 +0000 UTC m=+334.943009502" watchObservedRunningTime="2026-07-17 09:32:17.360373849 +0000 UTC m=+334.943012226"
Jul 17 09:32:18.196324 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:18.196224    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 09:32:18.196392 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:18.196363    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 09:32:18.339291 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:18.339254    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-96757" event={"ID":"e56c1862-83dd-417d-bd30-60abc092e276","Type":"ContainerStarted","Data":"d828d8ac2bccaf7dc492f26fbfbdacee89467cb6a2eddbfc8aed0f8c8fa7eff0"}
Jul 17 09:32:18.339744 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:18.339361    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-96757"
Jul 17 09:32:18.340926 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:18.340901    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s" event={"ID":"6add91c8-e0a7-4f12-9ce6-669a8bbbc160","Type":"ContainerStarted","Data":"d7e7e30d2d03e5faa96578bdb9ad932b47cbc3bb44a92bb0aff6a5a30605d993"}
Jul 17 09:32:18.341129 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:18.341115    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"
Jul 17 09:32:18.360135 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:18.360094    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-96757" podStartSLOduration=1.316263773 podStartE2EDuration="5.36008206s" totalImagesPullingTime="4.043818287s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:32:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:32:18.358004477 +0000 UTC m=+335.940642852" watchObservedRunningTime="2026-07-17 09:32:18.36008206 +0000 UTC m=+335.942720436"
Jul 17 09:32:18.382363 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:18.382327    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s" podStartSLOduration=0.970707194 podStartE2EDuration="5.382313537s" totalImagesPullingTime="4.411606343s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:32:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:32:18.381032843 +0000 UTC m=+335.963671231" watchObservedRunningTime="2026-07-17 09:32:18.382313537 +0000 UTC m=+335.964951913"
Jul 17 09:32:19.347323 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:19.347287    2574 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-qvd4s container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 17 09:32:19.347703 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:19.347363    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s" podUID="6add91c8-e0a7-4f12-9ce6-669a8bbbc160" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 09:32:22.346356 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:22.346321    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"
Jul 17 09:32:23.815506 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:23.815470    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-24c6h"]
Jul 17 09:32:23.821390 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:23.821371    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-24c6h"
Jul 17 09:32:23.824901 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:23.824880    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 17 09:32:23.825015 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:23.824879    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-zmmjw\""
Jul 17 09:32:23.825015 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:23.824932    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-v9rhs"]
Jul 17 09:32:23.829197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:23.829178    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-v9rhs"
Jul 17 09:32:23.830125 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:23.829531    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-24c6h"]
Jul 17 09:32:23.831784 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:23.831767    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 17 09:32:23.832224 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:23.832203    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-69td6\""
Jul 17 09:32:23.841900 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:23.841877    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-v9rhs"]
Jul 17 09:32:23.887964 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:23.887934    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/03e554d1-089a-4801-aa0c-ace3e09085a2-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-v9rhs\" (UID: \"03e554d1-089a-4801-aa0c-ace3e09085a2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-v9rhs"
Jul 17 09:32:23.888111 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:23.888003    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/75da319e-cb7b-471b-811c-cf9f9fbf120a-cert\") pod \"kserve-controller-manager-d6f4b65b8-24c6h\" (UID: \"75da319e-cb7b-471b-811c-cf9f9fbf120a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-24c6h"
Jul 17 09:32:23.888111 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:23.888035    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8zl8\" (UniqueName: \"kubernetes.io/projected/03e554d1-089a-4801-aa0c-ace3e09085a2-kube-api-access-d8zl8\") pod \"llmisvc-controller-manager-777c6bd7d5-v9rhs\" (UID: \"03e554d1-089a-4801-aa0c-ace3e09085a2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-v9rhs"
Jul 17 09:32:23.888111 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:23.888059    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwgsj\" (UniqueName: \"kubernetes.io/projected/75da319e-cb7b-471b-811c-cf9f9fbf120a-kube-api-access-lwgsj\") pod \"kserve-controller-manager-d6f4b65b8-24c6h\" (UID: \"75da319e-cb7b-471b-811c-cf9f9fbf120a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-24c6h"
Jul 17 09:32:23.989395 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:23.989364    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/03e554d1-089a-4801-aa0c-ace3e09085a2-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-v9rhs\" (UID: \"03e554d1-089a-4801-aa0c-ace3e09085a2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-v9rhs"
Jul 17 09:32:23.989546 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:23.989403    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/75da319e-cb7b-471b-811c-cf9f9fbf120a-cert\") pod \"kserve-controller-manager-d6f4b65b8-24c6h\" (UID: \"75da319e-cb7b-471b-811c-cf9f9fbf120a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-24c6h"
Jul 17 09:32:23.989546 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:32:23.989495    2574 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 17 09:32:23.989546 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:32:23.989539    2574 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75da319e-cb7b-471b-811c-cf9f9fbf120a-cert podName:75da319e-cb7b-471b-811c-cf9f9fbf120a nodeName:}" failed. No retries permitted until 2026-07-17 09:32:24.48952535 +0000 UTC m=+342.072163706 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/75da319e-cb7b-471b-811c-cf9f9fbf120a-cert") pod "kserve-controller-manager-d6f4b65b8-24c6h" (UID: "75da319e-cb7b-471b-811c-cf9f9fbf120a") : secret "kserve-webhook-server-cert" not found
Jul 17 09:32:23.989546 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:23.989536    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8zl8\" (UniqueName: \"kubernetes.io/projected/03e554d1-089a-4801-aa0c-ace3e09085a2-kube-api-access-d8zl8\") pod \"llmisvc-controller-manager-777c6bd7d5-v9rhs\" (UID: \"03e554d1-089a-4801-aa0c-ace3e09085a2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-v9rhs"
Jul 17 09:32:23.989759 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:23.989589    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwgsj\" (UniqueName: \"kubernetes.io/projected/75da319e-cb7b-471b-811c-cf9f9fbf120a-kube-api-access-lwgsj\") pod \"kserve-controller-manager-d6f4b65b8-24c6h\" (UID: \"75da319e-cb7b-471b-811c-cf9f9fbf120a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-24c6h"
Jul 17 09:32:23.991906 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:23.991884    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/03e554d1-089a-4801-aa0c-ace3e09085a2-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-v9rhs\" (UID: \"03e554d1-089a-4801-aa0c-ace3e09085a2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-v9rhs"
Jul 17 09:32:24.004631 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:24.004610    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8zl8\" (UniqueName: \"kubernetes.io/projected/03e554d1-089a-4801-aa0c-ace3e09085a2-kube-api-access-d8zl8\") pod \"llmisvc-controller-manager-777c6bd7d5-v9rhs\" (UID: \"03e554d1-089a-4801-aa0c-ace3e09085a2\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-v9rhs"
Jul 17 09:32:24.004992 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:24.004973    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwgsj\" (UniqueName: \"kubernetes.io/projected/75da319e-cb7b-471b-811c-cf9f9fbf120a-kube-api-access-lwgsj\") pod \"kserve-controller-manager-d6f4b65b8-24c6h\" (UID: \"75da319e-cb7b-471b-811c-cf9f9fbf120a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-24c6h"
Jul 17 09:32:24.143943 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:24.143853    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-v9rhs"
Jul 17 09:32:24.276630 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:24.276604    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-v9rhs"]
Jul 17 09:32:24.278523 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:32:24.278499    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod03e554d1_089a_4801_aa0c_ace3e09085a2.slice/crio-32e4bcc8540c4192b77d49a1aef341d6da35818e4d793185e672f0780ee7a11a WatchSource:0}: Error finding container 32e4bcc8540c4192b77d49a1aef341d6da35818e4d793185e672f0780ee7a11a: Status 404 returned error can't find the container with id 32e4bcc8540c4192b77d49a1aef341d6da35818e4d793185e672f0780ee7a11a
Jul 17 09:32:24.367088 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:24.367061    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-v9rhs" event={"ID":"03e554d1-089a-4801-aa0c-ace3e09085a2","Type":"ContainerStarted","Data":"32e4bcc8540c4192b77d49a1aef341d6da35818e4d793185e672f0780ee7a11a"}
Jul 17 09:32:24.495222 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:24.495181    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/75da319e-cb7b-471b-811c-cf9f9fbf120a-cert\") pod \"kserve-controller-manager-d6f4b65b8-24c6h\" (UID: \"75da319e-cb7b-471b-811c-cf9f9fbf120a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-24c6h"
Jul 17 09:32:24.497684 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:24.497665    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/75da319e-cb7b-471b-811c-cf9f9fbf120a-cert\") pod \"kserve-controller-manager-d6f4b65b8-24c6h\" (UID: \"75da319e-cb7b-471b-811c-cf9f9fbf120a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-24c6h"
Jul 17 09:32:24.734761 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:24.734728    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-24c6h"
Jul 17 09:32:24.872228 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:24.872061    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-24c6h"]
Jul 17 09:32:24.875059 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:32:24.875029    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75da319e_cb7b_471b_811c_cf9f9fbf120a.slice/crio-ef852d34dc3197ca789e16cc84f2c218f6b42dac7e5202206c71280833898580 WatchSource:0}: Error finding container ef852d34dc3197ca789e16cc84f2c218f6b42dac7e5202206c71280833898580: Status 404 returned error can't find the container with id ef852d34dc3197ca789e16cc84f2c218f6b42dac7e5202206c71280833898580
Jul 17 09:32:25.375369 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:25.375340    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-24c6h" event={"ID":"75da319e-cb7b-471b-811c-cf9f9fbf120a","Type":"ContainerStarted","Data":"ef852d34dc3197ca789e16cc84f2c218f6b42dac7e5202206c71280833898580"}
Jul 17 09:32:26.619004 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:26.618972    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"]
Jul 17 09:32:26.619446 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:26.619194    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s" podUID="6add91c8-e0a7-4f12-9ce6-669a8bbbc160" containerName="discovery" containerID="cri-o://d7e7e30d2d03e5faa96578bdb9ad932b47cbc3bb44a92bb0aff6a5a30605d993" gracePeriod=30
Jul 17 09:32:27.387566 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:27.387487    2574 generic.go:356] "Generic (PLEG): container finished" podID="6add91c8-e0a7-4f12-9ce6-669a8bbbc160" containerID="d7e7e30d2d03e5faa96578bdb9ad932b47cbc3bb44a92bb0aff6a5a30605d993" exitCode=0
Jul 17 09:32:27.387723 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:27.387570    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s" event={"ID":"6add91c8-e0a7-4f12-9ce6-669a8bbbc160","Type":"ContainerDied","Data":"d7e7e30d2d03e5faa96578bdb9ad932b47cbc3bb44a92bb0aff6a5a30605d993"}
Jul 17 09:32:27.578267 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:27.578222    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"
Jul 17 09:32:27.728484 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:27.728455    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-istio-csr-dns-cert\") pod \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\" (UID: \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\") "
Jul 17 09:32:27.728484 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:27.728488    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-local-certs\") pod \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\" (UID: \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\") "
Jul 17 09:32:27.728941 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:27.728508    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c9rzf\" (UniqueName: \"kubernetes.io/projected/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-kube-api-access-c9rzf\") pod \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\" (UID: \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\") "
Jul 17 09:32:27.728941 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:27.728528    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-istio-kubeconfig\") pod \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\" (UID: \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\") "
Jul 17 09:32:27.728941 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:27.728571    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-cacerts\") pod \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\" (UID: \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\") "
Jul 17 09:32:27.728941 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:27.728587    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-istio-token\") pod \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\" (UID: \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\") "
Jul 17 09:32:27.728941 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:27.728650    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-istio-csr-ca-configmap\") pod \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\" (UID: \"6add91c8-e0a7-4f12-9ce6-669a8bbbc160\") "
Jul 17 09:32:27.729292 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:27.729164    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "6add91c8-e0a7-4f12-9ce6-669a8bbbc160" (UID: "6add91c8-e0a7-4f12-9ce6-669a8bbbc160"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:32:27.731377 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:27.731345    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-kube-api-access-c9rzf" (OuterVolumeSpecName: "kube-api-access-c9rzf") pod "6add91c8-e0a7-4f12-9ce6-669a8bbbc160" (UID: "6add91c8-e0a7-4f12-9ce6-669a8bbbc160"). InnerVolumeSpecName "kube-api-access-c9rzf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:32:27.731489 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:27.731433    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "6add91c8-e0a7-4f12-9ce6-669a8bbbc160" (UID: "6add91c8-e0a7-4f12-9ce6-669a8bbbc160"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:32:27.731489 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:27.731452    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-cacerts" (OuterVolumeSpecName: "cacerts") pod "6add91c8-e0a7-4f12-9ce6-669a8bbbc160" (UID: "6add91c8-e0a7-4f12-9ce6-669a8bbbc160"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:32:27.731489 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:27.731463    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-istio-token" (OuterVolumeSpecName: "istio-token") pod "6add91c8-e0a7-4f12-9ce6-669a8bbbc160" (UID: "6add91c8-e0a7-4f12-9ce6-669a8bbbc160"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:32:27.731616 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:27.731508    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-local-certs" (OuterVolumeSpecName: "local-certs") pod "6add91c8-e0a7-4f12-9ce6-669a8bbbc160" (UID: "6add91c8-e0a7-4f12-9ce6-669a8bbbc160"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:32:27.731769 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:27.731744    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "6add91c8-e0a7-4f12-9ce6-669a8bbbc160" (UID: "6add91c8-e0a7-4f12-9ce6-669a8bbbc160"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:32:27.830151 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:27.830124    2574 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-istio-csr-ca-configmap\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:32:27.830151 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:27.830148    2574 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-istio-csr-dns-cert\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:32:27.830320 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:27.830157    2574 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-local-certs\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:32:27.830320 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:27.830167    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c9rzf\" (UniqueName: \"kubernetes.io/projected/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-kube-api-access-c9rzf\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:32:27.830320 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:27.830175    2574 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-istio-kubeconfig\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:32:27.830320 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:27.830183    2574 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-cacerts\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:32:27.830320 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:27.830192    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6add91c8-e0a7-4f12-9ce6-669a8bbbc160-istio-token\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:32:28.342759 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:28.342682    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-bs5g8"
Jul 17 09:32:28.395143 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:28.395106    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-v9rhs" event={"ID":"03e554d1-089a-4801-aa0c-ace3e09085a2","Type":"ContainerStarted","Data":"1b56777afffadc44ef4768fa72988eaa40ac19014cb560ebfc16b29542f0e1f2"}
Jul 17 09:32:28.395372 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:28.395353    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-v9rhs"
Jul 17 09:32:28.396967 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:28.396943    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-24c6h" event={"ID":"75da319e-cb7b-471b-811c-cf9f9fbf120a","Type":"ContainerStarted","Data":"ec1574a0a3ef4ee401fed96d1f937082622ee0e470b4233af8d21086cfcd929a"}
Jul 17 09:32:28.397079 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:28.397003    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-24c6h"
Jul 17 09:32:28.398521 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:28.398489    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s" event={"ID":"6add91c8-e0a7-4f12-9ce6-669a8bbbc160","Type":"ContainerDied","Data":"e67777653be4cff760a7bb5fd076964d1f61395344c5145fe370f339462f2ea3"}
Jul 17 09:32:28.398521 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:28.398514    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"
Jul 17 09:32:28.398648 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:28.398542    2574 scope.go:117] "RemoveContainer" containerID="d7e7e30d2d03e5faa96578bdb9ad932b47cbc3bb44a92bb0aff6a5a30605d993"
Jul 17 09:32:28.415323 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:28.415271    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-v9rhs" podStartSLOduration=2.0995181609999998 podStartE2EDuration="5.415234181s" totalImagesPullingTime="3.31571602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:32:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:32:28.413452652 +0000 UTC m=+345.996091029" watchObservedRunningTime="2026-07-17 09:32:28.415234181 +0000 UTC m=+345.997872558"
Jul 17 09:32:28.429649 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:28.429623    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"]
Jul 17 09:32:28.435946 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:28.435918    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-qvd4s"]
Jul 17 09:32:28.451638 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:28.451422    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-24c6h" podStartSLOduration=2.7360170889999997 podStartE2EDuration="5.451406505s" totalImagesPullingTime="2.715389416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:32:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:32:28.451263024 +0000 UTC m=+346.033901425" watchObservedRunningTime="2026-07-17 09:32:28.451406505 +0000 UTC m=+346.034044882"
Jul 17 09:32:28.897201 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:28.897169    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6add91c8-e0a7-4f12-9ce6-669a8bbbc160" path="/var/lib/kubelet/pods/6add91c8-e0a7-4f12-9ce6-669a8bbbc160/volumes"
Jul 17 09:32:29.349171 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:29.349142    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-96757"
Jul 17 09:32:59.405769 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:59.405739    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-v9rhs"
Jul 17 09:32:59.409506 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:32:59.409482    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-24c6h"
Jul 17 09:33:13.075393 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:13.075363    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wl69j"]
Jul 17 09:33:13.075725 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:13.075687    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6add91c8-e0a7-4f12-9ce6-669a8bbbc160" containerName="discovery"
Jul 17 09:33:13.075725 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:13.075697    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6add91c8-e0a7-4f12-9ce6-669a8bbbc160" containerName="discovery"
Jul 17 09:33:13.075807 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:13.075757    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6add91c8-e0a7-4f12-9ce6-669a8bbbc160" containerName="discovery"
Jul 17 09:33:13.077579 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:13.077564    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wl69j"
Jul 17 09:33:13.081746 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:13.081703    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 17 09:33:13.081874 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:13.081753    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 17 09:33:13.081874 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:13.081754    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-j5pv8\""
Jul 17 09:33:13.087924 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:13.087903    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wl69j"]
Jul 17 09:33:13.154701 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:13.154674    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzzr5\" (UniqueName: \"kubernetes.io/projected/e59aa8c4-c578-4237-a498-cad6b9a73ad5-kube-api-access-xzzr5\") pod \"kuadrant-operator-catalog-wl69j\" (UID: \"e59aa8c4-c578-4237-a498-cad6b9a73ad5\") " pod="kuadrant-system/kuadrant-operator-catalog-wl69j"
Jul 17 09:33:13.255551 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:13.255529    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzzr5\" (UniqueName: \"kubernetes.io/projected/e59aa8c4-c578-4237-a498-cad6b9a73ad5-kube-api-access-xzzr5\") pod \"kuadrant-operator-catalog-wl69j\" (UID: \"e59aa8c4-c578-4237-a498-cad6b9a73ad5\") " pod="kuadrant-system/kuadrant-operator-catalog-wl69j"
Jul 17 09:33:13.264164 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:13.264135    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzzr5\" (UniqueName: \"kubernetes.io/projected/e59aa8c4-c578-4237-a498-cad6b9a73ad5-kube-api-access-xzzr5\") pod \"kuadrant-operator-catalog-wl69j\" (UID: \"e59aa8c4-c578-4237-a498-cad6b9a73ad5\") " pod="kuadrant-system/kuadrant-operator-catalog-wl69j"
Jul 17 09:33:13.388936 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:13.388885    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wl69j"
Jul 17 09:33:13.444882 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:13.444849    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wl69j"]
Jul 17 09:33:13.517323 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:13.517299    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wl69j"]
Jul 17 09:33:13.519614 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:33:13.519587    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode59aa8c4_c578_4237_a498_cad6b9a73ad5.slice/crio-5369419d7a3ec8d6f7b2157a4ee4e6786ce912bd41f19131b74331c2c78f4a1a WatchSource:0}: Error finding container 5369419d7a3ec8d6f7b2157a4ee4e6786ce912bd41f19131b74331c2c78f4a1a: Status 404 returned error can't find the container with id 5369419d7a3ec8d6f7b2157a4ee4e6786ce912bd41f19131b74331c2c78f4a1a
Jul 17 09:33:13.599550 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:13.599523    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wl69j" event={"ID":"e59aa8c4-c578-4237-a498-cad6b9a73ad5","Type":"ContainerStarted","Data":"5369419d7a3ec8d6f7b2157a4ee4e6786ce912bd41f19131b74331c2c78f4a1a"}
Jul 17 09:33:13.643303 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:13.643217    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5x7hd"]
Jul 17 09:33:13.646050 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:13.646035    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5x7hd"
Jul 17 09:33:13.654033 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:13.654013    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5x7hd"]
Jul 17 09:33:13.657412 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:13.657391    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx9tf\" (UniqueName: \"kubernetes.io/projected/87acf608-ce98-43fc-820d-11da3b3cbc0d-kube-api-access-dx9tf\") pod \"kuadrant-operator-catalog-5x7hd\" (UID: \"87acf608-ce98-43fc-820d-11da3b3cbc0d\") " pod="kuadrant-system/kuadrant-operator-catalog-5x7hd"
Jul 17 09:33:13.757737 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:13.757714    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx9tf\" (UniqueName: \"kubernetes.io/projected/87acf608-ce98-43fc-820d-11da3b3cbc0d-kube-api-access-dx9tf\") pod \"kuadrant-operator-catalog-5x7hd\" (UID: \"87acf608-ce98-43fc-820d-11da3b3cbc0d\") " pod="kuadrant-system/kuadrant-operator-catalog-5x7hd"
Jul 17 09:33:13.767362 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:13.767339    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx9tf\" (UniqueName: \"kubernetes.io/projected/87acf608-ce98-43fc-820d-11da3b3cbc0d-kube-api-access-dx9tf\") pod \"kuadrant-operator-catalog-5x7hd\" (UID: \"87acf608-ce98-43fc-820d-11da3b3cbc0d\") " pod="kuadrant-system/kuadrant-operator-catalog-5x7hd"
Jul 17 09:33:13.955722 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:13.955692    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5x7hd"
Jul 17 09:33:14.101991 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:14.101964    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5x7hd"]
Jul 17 09:33:14.138842 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:33:14.138813    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87acf608_ce98_43fc_820d_11da3b3cbc0d.slice/crio-57d9c444c554687bfd3c73354a14616dea70fd7c4229ded8cebe75d26bab94d2 WatchSource:0}: Error finding container 57d9c444c554687bfd3c73354a14616dea70fd7c4229ded8cebe75d26bab94d2: Status 404 returned error can't find the container with id 57d9c444c554687bfd3c73354a14616dea70fd7c4229ded8cebe75d26bab94d2
Jul 17 09:33:14.606581 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:14.606537    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5x7hd" event={"ID":"87acf608-ce98-43fc-820d-11da3b3cbc0d","Type":"ContainerStarted","Data":"57d9c444c554687bfd3c73354a14616dea70fd7c4229ded8cebe75d26bab94d2"}
Jul 17 09:33:16.618380 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:16.618344    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wl69j" event={"ID":"e59aa8c4-c578-4237-a498-cad6b9a73ad5","Type":"ContainerStarted","Data":"2d2e4b52629721c438364519b1c93b78f53a682acbdb1fc6205e5d325b812c8e"}
Jul 17 09:33:16.618380 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:16.618367    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-wl69j" podUID="e59aa8c4-c578-4237-a498-cad6b9a73ad5" containerName="registry-server" containerID="cri-o://2d2e4b52629721c438364519b1c93b78f53a682acbdb1fc6205e5d325b812c8e" gracePeriod=2
Jul 17 09:33:16.619953 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:16.619918    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5x7hd" event={"ID":"87acf608-ce98-43fc-820d-11da3b3cbc0d","Type":"ContainerStarted","Data":"56e6b28eb4675429cd898a0f16535f6b0d0c6b99c086a51d18b41436283e85f0"}
Jul 17 09:33:16.635070 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:16.635030    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-wl69j" podStartSLOduration=1.418616168 podStartE2EDuration="3.635016415s" totalImagesPullingTime="2.216400247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:33:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:33:16.633600136 +0000 UTC m=+394.216238526" watchObservedRunningTime="2026-07-17 09:33:16.635016415 +0000 UTC m=+394.217654790"
Jul 17 09:33:16.650526 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:16.650490    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-5x7hd" podStartSLOduration=1.947352325 podStartE2EDuration="3.650476579s" totalImagesPullingTime="1.703124254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:33:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:33:16.649375158 +0000 UTC m=+394.232013534" watchObservedRunningTime="2026-07-17 09:33:16.650476579 +0000 UTC m=+394.233114955"
Jul 17 09:33:16.869572 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:16.869519    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wl69j"
Jul 17 09:33:16.883872 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:16.883843    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xzzr5\" (UniqueName: \"kubernetes.io/projected/e59aa8c4-c578-4237-a498-cad6b9a73ad5-kube-api-access-xzzr5\") pod \"e59aa8c4-c578-4237-a498-cad6b9a73ad5\" (UID: \"e59aa8c4-c578-4237-a498-cad6b9a73ad5\") "
Jul 17 09:33:16.886544 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:16.886514    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e59aa8c4-c578-4237-a498-cad6b9a73ad5-kube-api-access-xzzr5" (OuterVolumeSpecName: "kube-api-access-xzzr5") pod "e59aa8c4-c578-4237-a498-cad6b9a73ad5" (UID: "e59aa8c4-c578-4237-a498-cad6b9a73ad5"). InnerVolumeSpecName "kube-api-access-xzzr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:33:16.984744 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:16.984723    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzzr5\" (UniqueName: \"kubernetes.io/projected/e59aa8c4-c578-4237-a498-cad6b9a73ad5-kube-api-access-xzzr5\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:33:17.625390 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:17.625358    2574 generic.go:356] "Generic (PLEG): container finished" podID="e59aa8c4-c578-4237-a498-cad6b9a73ad5" containerID="2d2e4b52629721c438364519b1c93b78f53a682acbdb1fc6205e5d325b812c8e" exitCode=0
Jul 17 09:33:17.625789 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:17.625402    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wl69j"
Jul 17 09:33:17.625789 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:17.625435    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wl69j" event={"ID":"e59aa8c4-c578-4237-a498-cad6b9a73ad5","Type":"ContainerDied","Data":"2d2e4b52629721c438364519b1c93b78f53a682acbdb1fc6205e5d325b812c8e"}
Jul 17 09:33:17.625789 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:17.625473    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wl69j" event={"ID":"e59aa8c4-c578-4237-a498-cad6b9a73ad5","Type":"ContainerDied","Data":"5369419d7a3ec8d6f7b2157a4ee4e6786ce912bd41f19131b74331c2c78f4a1a"}
Jul 17 09:33:17.625789 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:17.625492    2574 scope.go:117] "RemoveContainer" containerID="2d2e4b52629721c438364519b1c93b78f53a682acbdb1fc6205e5d325b812c8e"
Jul 17 09:33:17.634259 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:17.634225    2574 scope.go:117] "RemoveContainer" containerID="2d2e4b52629721c438364519b1c93b78f53a682acbdb1fc6205e5d325b812c8e"
Jul 17 09:33:17.634502 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:33:17.634485    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d2e4b52629721c438364519b1c93b78f53a682acbdb1fc6205e5d325b812c8e\": container with ID starting with 2d2e4b52629721c438364519b1c93b78f53a682acbdb1fc6205e5d325b812c8e not found: ID does not exist" containerID="2d2e4b52629721c438364519b1c93b78f53a682acbdb1fc6205e5d325b812c8e"
Jul 17 09:33:17.634556 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:17.634509    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d2e4b52629721c438364519b1c93b78f53a682acbdb1fc6205e5d325b812c8e"} err="failed to get container status \"2d2e4b52629721c438364519b1c93b78f53a682acbdb1fc6205e5d325b812c8e\": rpc error: code = NotFound desc = could not find container \"2d2e4b52629721c438364519b1c93b78f53a682acbdb1fc6205e5d325b812c8e\": container with ID starting with 2d2e4b52629721c438364519b1c93b78f53a682acbdb1fc6205e5d325b812c8e not found: ID does not exist"
Jul 17 09:33:17.642655 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:17.642626    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wl69j"]
Jul 17 09:33:17.645321 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:17.645300    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wl69j"]
Jul 17 09:33:18.897076 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:18.897045    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e59aa8c4-c578-4237-a498-cad6b9a73ad5" path="/var/lib/kubelet/pods/e59aa8c4-c578-4237-a498-cad6b9a73ad5/volumes"
Jul 17 09:33:23.955871 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:23.955842    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-5x7hd"
Jul 17 09:33:23.956224 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:23.955888    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-5x7hd"
Jul 17 09:33:23.978212 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:23.978192    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-5x7hd"
Jul 17 09:33:24.678917 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:24.678890    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-5x7hd"
Jul 17 09:33:47.207306 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:47.207272    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx"]
Jul 17 09:33:47.207839 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:47.207793    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e59aa8c4-c578-4237-a498-cad6b9a73ad5" containerName="registry-server"
Jul 17 09:33:47.207839 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:47.207812    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e59aa8c4-c578-4237-a498-cad6b9a73ad5" containerName="registry-server"
Jul 17 09:33:47.207957 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:47.207892    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e59aa8c4-c578-4237-a498-cad6b9a73ad5" containerName="registry-server"
Jul 17 09:33:47.216645 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:47.216622    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx"
Jul 17 09:33:47.219442 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:47.219423    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-h2vwj\""
Jul 17 09:33:47.228628 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:47.228600    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx"]
Jul 17 09:33:47.303389 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:47.303363    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rt5r2\" (UniqueName: \"kubernetes.io/projected/2619147d-4fbd-4688-8692-eaaab0b68f33-kube-api-access-rt5r2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rf5hx\" (UID: \"2619147d-4fbd-4688-8692-eaaab0b68f33\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx"
Jul 17 09:33:47.303485 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:47.303468    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2619147d-4fbd-4688-8692-eaaab0b68f33-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rf5hx\" (UID: \"2619147d-4fbd-4688-8692-eaaab0b68f33\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx"
Jul 17 09:33:47.404775 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:47.404697    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rt5r2\" (UniqueName: \"kubernetes.io/projected/2619147d-4fbd-4688-8692-eaaab0b68f33-kube-api-access-rt5r2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rf5hx\" (UID: \"2619147d-4fbd-4688-8692-eaaab0b68f33\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx"
Jul 17 09:33:47.404933 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:47.404826    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2619147d-4fbd-4688-8692-eaaab0b68f33-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rf5hx\" (UID: \"2619147d-4fbd-4688-8692-eaaab0b68f33\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx"
Jul 17 09:33:47.405320 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:47.405289    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2619147d-4fbd-4688-8692-eaaab0b68f33-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rf5hx\" (UID: \"2619147d-4fbd-4688-8692-eaaab0b68f33\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx"
Jul 17 09:33:47.421178 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:47.421149    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rt5r2\" (UniqueName: \"kubernetes.io/projected/2619147d-4fbd-4688-8692-eaaab0b68f33-kube-api-access-rt5r2\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rf5hx\" (UID: \"2619147d-4fbd-4688-8692-eaaab0b68f33\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx"
Jul 17 09:33:47.527434 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:47.527367    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx"
Jul 17 09:33:47.655680 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:47.655651    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx"]
Jul 17 09:33:47.657301 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:33:47.657276    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2619147d_4fbd_4688_8692_eaaab0b68f33.slice/crio-50d93d94de8ecda46466be8b4edde5ebc16823d7afeac744227a57e9ba291ebb WatchSource:0}: Error finding container 50d93d94de8ecda46466be8b4edde5ebc16823d7afeac744227a57e9ba291ebb: Status 404 returned error can't find the container with id 50d93d94de8ecda46466be8b4edde5ebc16823d7afeac744227a57e9ba291ebb
Jul 17 09:33:47.765107 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:47.765079    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx" event={"ID":"2619147d-4fbd-4688-8692-eaaab0b68f33","Type":"ContainerStarted","Data":"50d93d94de8ecda46466be8b4edde5ebc16823d7afeac744227a57e9ba291ebb"}
Jul 17 09:33:49.824138 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:49.824103    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-mk4bq"]
Jul 17 09:33:49.828057 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:49.828040    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mk4bq"
Jul 17 09:33:49.831855 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:49.831837    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-wdcvf\""
Jul 17 09:33:49.839722 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:49.839703    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-mk4bq"]
Jul 17 09:33:49.923573 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:49.923547    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l584l\" (UniqueName: \"kubernetes.io/projected/8e21489e-210e-4b51-9777-e637fb5cfc74-kube-api-access-l584l\") pod \"authorino-operator-75cc8d5cb7-mk4bq\" (UID: \"8e21489e-210e-4b51-9777-e637fb5cfc74\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-mk4bq"
Jul 17 09:33:50.024146 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:50.024115    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l584l\" (UniqueName: \"kubernetes.io/projected/8e21489e-210e-4b51-9777-e637fb5cfc74-kube-api-access-l584l\") pod \"authorino-operator-75cc8d5cb7-mk4bq\" (UID: \"8e21489e-210e-4b51-9777-e637fb5cfc74\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-mk4bq"
Jul 17 09:33:50.037770 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:50.037747    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l584l\" (UniqueName: \"kubernetes.io/projected/8e21489e-210e-4b51-9777-e637fb5cfc74-kube-api-access-l584l\") pod \"authorino-operator-75cc8d5cb7-mk4bq\" (UID: \"8e21489e-210e-4b51-9777-e637fb5cfc74\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-mk4bq"
Jul 17 09:33:50.138838 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:50.138779    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mk4bq"
Jul 17 09:33:50.291054 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:50.291034    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-mk4bq"]
Jul 17 09:33:50.293188 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:33:50.293158    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e21489e_210e_4b51_9777_e637fb5cfc74.slice/crio-5b10f813b6625152c53e05bf462054fc8ff4efd8aa3f1d32532432613a6e6704 WatchSource:0}: Error finding container 5b10f813b6625152c53e05bf462054fc8ff4efd8aa3f1d32532432613a6e6704: Status 404 returned error can't find the container with id 5b10f813b6625152c53e05bf462054fc8ff4efd8aa3f1d32532432613a6e6704
Jul 17 09:33:50.780298 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:50.780263    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mk4bq" event={"ID":"8e21489e-210e-4b51-9777-e637fb5cfc74","Type":"ContainerStarted","Data":"5b10f813b6625152c53e05bf462054fc8ff4efd8aa3f1d32532432613a6e6704"}
Jul 17 09:33:51.965348 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:51.965318    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7464459688-4rdzs"]
Jul 17 09:33:51.972197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:51.972171    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7464459688-4rdzs"
Jul 17 09:33:51.976930 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:51.976903    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 17 09:33:51.977065 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:51.976937    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 17 09:33:51.978423 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:51.978403    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-692fj\""
Jul 17 09:33:51.979457 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:51.979303    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 17 09:33:51.979457 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:51.979414    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 17 09:33:51.979685 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:51.979458    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 17 09:33:51.983212 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:51.983188    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 17 09:33:51.983931 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:51.983879    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7464459688-4rdzs"]
Jul 17 09:33:52.042851 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.042826    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5c2d4229-4cd7-4951-b0d3-43441e51732d-console-config\") pod \"console-7464459688-4rdzs\" (UID: \"5c2d4229-4cd7-4951-b0d3-43441e51732d\") " pod="openshift-console/console-7464459688-4rdzs"
Jul 17 09:33:52.042990 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.042878    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5c2d4229-4cd7-4951-b0d3-43441e51732d-console-oauth-config\") pod \"console-7464459688-4rdzs\" (UID: \"5c2d4229-4cd7-4951-b0d3-43441e51732d\") " pod="openshift-console/console-7464459688-4rdzs"
Jul 17 09:33:52.042990 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.042905    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5c2d4229-4cd7-4951-b0d3-43441e51732d-console-serving-cert\") pod \"console-7464459688-4rdzs\" (UID: \"5c2d4229-4cd7-4951-b0d3-43441e51732d\") " pod="openshift-console/console-7464459688-4rdzs"
Jul 17 09:33:52.042990 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.042976    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5c2d4229-4cd7-4951-b0d3-43441e51732d-service-ca\") pod \"console-7464459688-4rdzs\" (UID: \"5c2d4229-4cd7-4951-b0d3-43441e51732d\") " pod="openshift-console/console-7464459688-4rdzs"
Jul 17 09:33:52.043163 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.043105    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c2d4229-4cd7-4951-b0d3-43441e51732d-trusted-ca-bundle\") pod \"console-7464459688-4rdzs\" (UID: \"5c2d4229-4cd7-4951-b0d3-43441e51732d\") " pod="openshift-console/console-7464459688-4rdzs"
Jul 17 09:33:52.043163 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.043144    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7mwrt\" (UniqueName: \"kubernetes.io/projected/5c2d4229-4cd7-4951-b0d3-43441e51732d-kube-api-access-7mwrt\") pod \"console-7464459688-4rdzs\" (UID: \"5c2d4229-4cd7-4951-b0d3-43441e51732d\") " pod="openshift-console/console-7464459688-4rdzs"
Jul 17 09:33:52.043295 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.043169    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5c2d4229-4cd7-4951-b0d3-43441e51732d-oauth-serving-cert\") pod \"console-7464459688-4rdzs\" (UID: \"5c2d4229-4cd7-4951-b0d3-43441e51732d\") " pod="openshift-console/console-7464459688-4rdzs"
Jul 17 09:33:52.044378 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.044359    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-zvqp9"]
Jul 17 09:33:52.048302 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.048281    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zvqp9"
Jul 17 09:33:52.051356 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.051336    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 17 09:33:52.051459 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.051387    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-m5c97\""
Jul 17 09:33:52.062030 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.062009    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-zvqp9"]
Jul 17 09:33:52.144492 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.144464    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c2d4229-4cd7-4951-b0d3-43441e51732d-trusted-ca-bundle\") pod \"console-7464459688-4rdzs\" (UID: \"5c2d4229-4cd7-4951-b0d3-43441e51732d\") " pod="openshift-console/console-7464459688-4rdzs"
Jul 17 09:33:52.144667 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.144512    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7mwrt\" (UniqueName: \"kubernetes.io/projected/5c2d4229-4cd7-4951-b0d3-43441e51732d-kube-api-access-7mwrt\") pod \"console-7464459688-4rdzs\" (UID: \"5c2d4229-4cd7-4951-b0d3-43441e51732d\") " pod="openshift-console/console-7464459688-4rdzs"
Jul 17 09:33:52.144667 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.144535    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5c2d4229-4cd7-4951-b0d3-43441e51732d-oauth-serving-cert\") pod \"console-7464459688-4rdzs\" (UID: \"5c2d4229-4cd7-4951-b0d3-43441e51732d\") " pod="openshift-console/console-7464459688-4rdzs"
Jul 17 09:33:52.144667 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.144578    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5c2d4229-4cd7-4951-b0d3-43441e51732d-console-config\") pod \"console-7464459688-4rdzs\" (UID: \"5c2d4229-4cd7-4951-b0d3-43441e51732d\") " pod="openshift-console/console-7464459688-4rdzs"
Jul 17 09:33:52.144667 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.144613    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5c2d4229-4cd7-4951-b0d3-43441e51732d-console-oauth-config\") pod \"console-7464459688-4rdzs\" (UID: \"5c2d4229-4cd7-4951-b0d3-43441e51732d\") " pod="openshift-console/console-7464459688-4rdzs"
Jul 17 09:33:52.144667 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.144637    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5c2d4229-4cd7-4951-b0d3-43441e51732d-console-serving-cert\") pod \"console-7464459688-4rdzs\" (UID: \"5c2d4229-4cd7-4951-b0d3-43441e51732d\") " pod="openshift-console/console-7464459688-4rdzs"
Jul 17 09:33:52.144930 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.144671    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gq27c\" (UniqueName: \"kubernetes.io/projected/0e5c29c3-e1f5-48b8-a3a1-f3579196c61c-kube-api-access-gq27c\") pod \"dns-operator-controller-manager-864dd78c4c-zvqp9\" (UID: \"0e5c29c3-e1f5-48b8-a3a1-f3579196c61c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zvqp9"
Jul 17 09:33:52.144930 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.144705    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5c2d4229-4cd7-4951-b0d3-43441e51732d-service-ca\") pod \"console-7464459688-4rdzs\" (UID: \"5c2d4229-4cd7-4951-b0d3-43441e51732d\") " pod="openshift-console/console-7464459688-4rdzs"
Jul 17 09:33:52.145410 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.145386    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5c2d4229-4cd7-4951-b0d3-43441e51732d-oauth-serving-cert\") pod \"console-7464459688-4rdzs\" (UID: \"5c2d4229-4cd7-4951-b0d3-43441e51732d\") " pod="openshift-console/console-7464459688-4rdzs"
Jul 17 09:33:52.145516 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.145415    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5c2d4229-4cd7-4951-b0d3-43441e51732d-service-ca\") pod \"console-7464459688-4rdzs\" (UID: \"5c2d4229-4cd7-4951-b0d3-43441e51732d\") " pod="openshift-console/console-7464459688-4rdzs"
Jul 17 09:33:52.145751 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.145650    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5c2d4229-4cd7-4951-b0d3-43441e51732d-console-config\") pod \"console-7464459688-4rdzs\" (UID: \"5c2d4229-4cd7-4951-b0d3-43441e51732d\") " pod="openshift-console/console-7464459688-4rdzs"
Jul 17 09:33:52.145904 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.145817    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5c2d4229-4cd7-4951-b0d3-43441e51732d-trusted-ca-bundle\") pod \"console-7464459688-4rdzs\" (UID: \"5c2d4229-4cd7-4951-b0d3-43441e51732d\") " pod="openshift-console/console-7464459688-4rdzs"
Jul 17 09:33:52.147744 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.147722    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5c2d4229-4cd7-4951-b0d3-43441e51732d-console-oauth-config\") pod \"console-7464459688-4rdzs\" (UID: \"5c2d4229-4cd7-4951-b0d3-43441e51732d\") " pod="openshift-console/console-7464459688-4rdzs"
Jul 17 09:33:52.148143 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.148120    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5c2d4229-4cd7-4951-b0d3-43441e51732d-console-serving-cert\") pod \"console-7464459688-4rdzs\" (UID: \"5c2d4229-4cd7-4951-b0d3-43441e51732d\") " pod="openshift-console/console-7464459688-4rdzs"
Jul 17 09:33:52.153077 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.153052    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7mwrt\" (UniqueName: \"kubernetes.io/projected/5c2d4229-4cd7-4951-b0d3-43441e51732d-kube-api-access-7mwrt\") pod \"console-7464459688-4rdzs\" (UID: \"5c2d4229-4cd7-4951-b0d3-43441e51732d\") " pod="openshift-console/console-7464459688-4rdzs"
Jul 17 09:33:52.245906 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.245825    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq27c\" (UniqueName: \"kubernetes.io/projected/0e5c29c3-e1f5-48b8-a3a1-f3579196c61c-kube-api-access-gq27c\") pod \"dns-operator-controller-manager-864dd78c4c-zvqp9\" (UID: \"0e5c29c3-e1f5-48b8-a3a1-f3579196c61c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zvqp9"
Jul 17 09:33:52.257761 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.257725    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gq27c\" (UniqueName: \"kubernetes.io/projected/0e5c29c3-e1f5-48b8-a3a1-f3579196c61c-kube-api-access-gq27c\") pod \"dns-operator-controller-manager-864dd78c4c-zvqp9\" (UID: \"0e5c29c3-e1f5-48b8-a3a1-f3579196c61c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zvqp9"
Jul 17 09:33:52.284719 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.284685    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7464459688-4rdzs"
Jul 17 09:33:52.360902 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:52.360870    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zvqp9"
Jul 17 09:33:54.155116 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:54.155088    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7464459688-4rdzs"]
Jul 17 09:33:54.157078 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:33:54.157050    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c2d4229_4cd7_4951_b0d3_43441e51732d.slice/crio-c1ae4274db9c8e757a2be9eff4c0cf0ee8b1fe476474032744e6044ca9280c8f WatchSource:0}: Error finding container c1ae4274db9c8e757a2be9eff4c0cf0ee8b1fe476474032744e6044ca9280c8f: Status 404 returned error can't find the container with id c1ae4274db9c8e757a2be9eff4c0cf0ee8b1fe476474032744e6044ca9280c8f
Jul 17 09:33:54.179156 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:54.179132    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-zvqp9"]
Jul 17 09:33:54.188555 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:33:54.188534    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e5c29c3_e1f5_48b8_a3a1_f3579196c61c.slice/crio-cf977bb6446932628439c6099e5272544f5e470ad78a3647e51fb1713482e94c WatchSource:0}: Error finding container cf977bb6446932628439c6099e5272544f5e470ad78a3647e51fb1713482e94c: Status 404 returned error can't find the container with id cf977bb6446932628439c6099e5272544f5e470ad78a3647e51fb1713482e94c
Jul 17 09:33:54.368579 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:54.368505    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5ts5r"]
Jul 17 09:33:54.371905 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:54.371886    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5ts5r"
Jul 17 09:33:54.376128 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:54.376111    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-m4jfp\""
Jul 17 09:33:54.396849 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:54.396826    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5ts5r"]
Jul 17 09:33:54.465894 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:54.465868    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfb49\" (UniqueName: \"kubernetes.io/projected/100068db-bcc8-421c-81cc-736382c28819-kube-api-access-gfb49\") pod \"limitador-operator-controller-manager-859d7b78bd-5ts5r\" (UID: \"100068db-bcc8-421c-81cc-736382c28819\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5ts5r"
Jul 17 09:33:54.566930 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:54.566901    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfb49\" (UniqueName: \"kubernetes.io/projected/100068db-bcc8-421c-81cc-736382c28819-kube-api-access-gfb49\") pod \"limitador-operator-controller-manager-859d7b78bd-5ts5r\" (UID: \"100068db-bcc8-421c-81cc-736382c28819\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5ts5r"
Jul 17 09:33:54.579032 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:54.579001    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfb49\" (UniqueName: \"kubernetes.io/projected/100068db-bcc8-421c-81cc-736382c28819-kube-api-access-gfb49\") pod \"limitador-operator-controller-manager-859d7b78bd-5ts5r\" (UID: \"100068db-bcc8-421c-81cc-736382c28819\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5ts5r"
Jul 17 09:33:54.681718 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:54.681697    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5ts5r"
Jul 17 09:33:54.806476 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:54.806439    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mk4bq" event={"ID":"8e21489e-210e-4b51-9777-e637fb5cfc74","Type":"ContainerStarted","Data":"ea620153d2eaad61582a50f8c6cf015318768543dc32e483e19983c65ca1a27e"}
Jul 17 09:33:54.806689 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:54.806502    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mk4bq"
Jul 17 09:33:54.807835 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:54.807812    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zvqp9" event={"ID":"0e5c29c3-e1f5-48b8-a3a1-f3579196c61c","Type":"ContainerStarted","Data":"cf977bb6446932628439c6099e5272544f5e470ad78a3647e51fb1713482e94c"}
Jul 17 09:33:54.809521 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:54.809497    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx" event={"ID":"2619147d-4fbd-4688-8692-eaaab0b68f33","Type":"ContainerStarted","Data":"219795f7e23341dfbf4c6a0e1e5b9071dfac9dce07dde5e38f24288228e61456"}
Jul 17 09:33:54.809620 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:54.809591    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx"
Jul 17 09:33:54.811144 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:54.811122    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7464459688-4rdzs" event={"ID":"5c2d4229-4cd7-4951-b0d3-43441e51732d","Type":"ContainerStarted","Data":"c173098d21f2050c5f2f5bd72672a763a124d1df018abc1cbf9d70984ce705ce"}
Jul 17 09:33:54.811226 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:54.811160    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7464459688-4rdzs" event={"ID":"5c2d4229-4cd7-4951-b0d3-43441e51732d","Type":"ContainerStarted","Data":"c1ae4274db9c8e757a2be9eff4c0cf0ee8b1fe476474032744e6044ca9280c8f"}
Jul 17 09:33:54.815290 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:54.815270    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5ts5r"]
Jul 17 09:33:54.817272 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:33:54.817234    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod100068db_bcc8_421c_81cc_736382c28819.slice/crio-fc05d4e6305ddf44e42e08c31bece61c5adbf0aad402ea26eb47fa905a4757c5 WatchSource:0}: Error finding container fc05d4e6305ddf44e42e08c31bece61c5adbf0aad402ea26eb47fa905a4757c5: Status 404 returned error can't find the container with id fc05d4e6305ddf44e42e08c31bece61c5adbf0aad402ea26eb47fa905a4757c5
Jul 17 09:33:54.827744 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:54.827707    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mk4bq" podStartSLOduration=2.120284942 podStartE2EDuration="5.827695111s" totalImagesPullingTime="3.707410169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:33:54.826332799 +0000 UTC m=+432.408971176" watchObservedRunningTime="2026-07-17 09:33:54.827695111 +0000 UTC m=+432.410333465"
Jul 17 09:33:54.847784 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:54.847742    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7464459688-4rdzs" podStartSLOduration=3.8477288229999997 podStartE2EDuration="3.847728823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:33:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:33:54.846372989 +0000 UTC m=+432.429011365" watchObservedRunningTime="2026-07-17 09:33:54.847728823 +0000 UTC m=+432.430367198"
Jul 17 09:33:54.869202 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:54.868814    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx" podStartSLOduration=1.508788424 podStartE2EDuration="7.868795705s" totalImagesPullingTime="6.360007281s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:33:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:33:54.866685486 +0000 UTC m=+432.449323887" watchObservedRunningTime="2026-07-17 09:33:54.868795705 +0000 UTC m=+432.451434081"
Jul 17 09:33:55.817948 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:55.817911    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5ts5r" event={"ID":"100068db-bcc8-421c-81cc-736382c28819","Type":"ContainerStarted","Data":"fc05d4e6305ddf44e42e08c31bece61c5adbf0aad402ea26eb47fa905a4757c5"}
Jul 17 09:33:56.824387 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:56.824354    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zvqp9" event={"ID":"0e5c29c3-e1f5-48b8-a3a1-f3579196c61c","Type":"ContainerStarted","Data":"07e6abc995945db3b7296248e58e3ac6bd318d034fb0af4ee0ceaca9af90bf5b"}
Jul 17 09:33:56.824710 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:56.824453    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zvqp9"
Jul 17 09:33:56.825964 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:56.825940    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5ts5r" event={"ID":"100068db-bcc8-421c-81cc-736382c28819","Type":"ContainerStarted","Data":"99b323499fb155147a641359f4e90c17a36e0e1a360405cc4c0037cb5f818b9d"}
Jul 17 09:33:56.826080 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:56.826063    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5ts5r"
Jul 17 09:33:56.843903 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:56.843861    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zvqp9" podStartSLOduration=2.755094481 podStartE2EDuration="4.843847635s" totalImagesPullingTime="2.088753154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:33:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:33:56.84253057 +0000 UTC m=+434.425168947" watchObservedRunningTime="2026-07-17 09:33:56.843847635 +0000 UTC m=+434.426486075"
Jul 17 09:33:56.860037 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:33:56.860001    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5ts5r" podStartSLOduration=0.9897367 podStartE2EDuration="2.859989699s" totalImagesPullingTime="1.870252999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:33:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:33:56.858325065 +0000 UTC m=+434.440963442" watchObservedRunningTime="2026-07-17 09:33:56.859989699 +0000 UTC m=+434.442628075"
Jul 17 09:34:02.285715 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:02.285684    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7464459688-4rdzs"
Jul 17 09:34:02.286090 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:02.285736    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7464459688-4rdzs"
Jul 17 09:34:02.290197 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:02.290173    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7464459688-4rdzs"
Jul 17 09:34:02.858896 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:02.858870    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7464459688-4rdzs"
Jul 17 09:34:05.820255 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:05.820158    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx"
Jul 17 09:34:05.820255 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:05.820220    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mk4bq"
Jul 17 09:34:07.573482 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.573454    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx"]
Jul 17 09:34:07.573897 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.573681    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx" podUID="2619147d-4fbd-4688-8692-eaaab0b68f33" containerName="manager" containerID="cri-o://219795f7e23341dfbf4c6a0e1e5b9071dfac9dce07dde5e38f24288228e61456" gracePeriod=2
Jul 17 09:34:07.580847 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.580820    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx"]
Jul 17 09:34:07.585975 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:34:07.585942    2574 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"2619147d-4fbd-4688-8692-eaaab0b68f33\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-rf5hx\": pods \"kuadrant-operator-controller-manager-8684c9ff49-rf5hx\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx"
Jul 17 09:34:07.595325 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.595297    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nql6d"]
Jul 17 09:34:07.595815 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.595795    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2619147d-4fbd-4688-8692-eaaab0b68f33" containerName="manager"
Jul 17 09:34:07.595815 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.595816    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2619147d-4fbd-4688-8692-eaaab0b68f33" containerName="manager"
Jul 17 09:34:07.595957 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.595900    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2619147d-4fbd-4688-8692-eaaab0b68f33" containerName="manager"
Jul 17 09:34:07.598631 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.598611    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5ts5r"]
Jul 17 09:34:07.598752 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.598728    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nql6d"
Jul 17 09:34:07.598877 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.598806    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5ts5r" podUID="100068db-bcc8-421c-81cc-736382c28819" containerName="manager" containerID="cri-o://99b323499fb155147a641359f4e90c17a36e0e1a360405cc4c0037cb5f818b9d" gracePeriod=2
Jul 17 09:34:07.599603 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.599127    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5ts5r"]
Jul 17 09:34:07.601300 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:34:07.601274    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rf5hx\" is forbidden: User \"system:node:ip-10-0-136-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-235.ec2.internal' and this object" podUID="2619147d-4fbd-4688-8692-eaaab0b68f33" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx"
Jul 17 09:34:07.603314 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:34:07.603287    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5ts5r\" is forbidden: User \"system:node:ip-10-0-136-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-235.ec2.internal' and this object" podUID="100068db-bcc8-421c-81cc-736382c28819" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5ts5r"
Jul 17 09:34:07.605647 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:34:07.605616    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rf5hx\" is forbidden: User \"system:node:ip-10-0-136-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-235.ec2.internal' and this object" podUID="2619147d-4fbd-4688-8692-eaaab0b68f33" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx"
Jul 17 09:34:07.608687 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:34:07.608663    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5ts5r\" is forbidden: User \"system:node:ip-10-0-136-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-235.ec2.internal' and this object" podUID="100068db-bcc8-421c-81cc-736382c28819" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5ts5r"
Jul 17 09:34:07.613142 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.613119    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nql6d"]
Jul 17 09:34:07.623258 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.623217    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hrbff"]
Jul 17 09:34:07.623608 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.623595    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="100068db-bcc8-421c-81cc-736382c28819" containerName="manager"
Jul 17 09:34:07.623649 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.623611    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="100068db-bcc8-421c-81cc-736382c28819" containerName="manager"
Jul 17 09:34:07.623690 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.623681    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="100068db-bcc8-421c-81cc-736382c28819" containerName="manager"
Jul 17 09:34:07.626335 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.626320    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hrbff"
Jul 17 09:34:07.632846 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:34:07.632817    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rf5hx\" is forbidden: User \"system:node:ip-10-0-136-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-235.ec2.internal' and this object" podUID="2619147d-4fbd-4688-8692-eaaab0b68f33" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx"
Jul 17 09:34:07.636310 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:34:07.636279    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5ts5r\" is forbidden: User \"system:node:ip-10-0-136-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-235.ec2.internal' and this object" podUID="100068db-bcc8-421c-81cc-736382c28819" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5ts5r"
Jul 17 09:34:07.638660 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.638636    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hrbff"]
Jul 17 09:34:07.768218 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.768190    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b11d398a-eb1b-4299-b1b1-85059ad5e910-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nql6d\" (UID: \"b11d398a-eb1b-4299-b1b1-85059ad5e910\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nql6d"
Jul 17 09:34:07.768419 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.768323    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdx8r\" (UniqueName: \"kubernetes.io/projected/b11d398a-eb1b-4299-b1b1-85059ad5e910-kube-api-access-pdx8r\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nql6d\" (UID: \"b11d398a-eb1b-4299-b1b1-85059ad5e910\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nql6d"
Jul 17 09:34:07.768419 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.768369    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm9st\" (UniqueName: \"kubernetes.io/projected/7eceeaf3-ab95-4000-913c-2254baf18b00-kube-api-access-mm9st\") pod \"limitador-operator-controller-manager-859d7b78bd-hrbff\" (UID: \"7eceeaf3-ab95-4000-913c-2254baf18b00\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hrbff"
Jul 17 09:34:07.833272 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.833214    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-zvqp9"
Jul 17 09:34:07.836174 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:34:07.836142    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rf5hx\" is forbidden: User \"system:node:ip-10-0-136-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-235.ec2.internal' and this object" podUID="2619147d-4fbd-4688-8692-eaaab0b68f33" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx"
Jul 17 09:34:07.853906 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.853883    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx"
Jul 17 09:34:07.856883 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.856864    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5ts5r"
Jul 17 09:34:07.869493 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.869475    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b11d398a-eb1b-4299-b1b1-85059ad5e910-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nql6d\" (UID: \"b11d398a-eb1b-4299-b1b1-85059ad5e910\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nql6d"
Jul 17 09:34:07.869583 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.869529    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdx8r\" (UniqueName: \"kubernetes.io/projected/b11d398a-eb1b-4299-b1b1-85059ad5e910-kube-api-access-pdx8r\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nql6d\" (UID: \"b11d398a-eb1b-4299-b1b1-85059ad5e910\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nql6d"
Jul 17 09:34:07.869658 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.869638    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm9st\" (UniqueName: \"kubernetes.io/projected/7eceeaf3-ab95-4000-913c-2254baf18b00-kube-api-access-mm9st\") pod \"limitador-operator-controller-manager-859d7b78bd-hrbff\" (UID: \"7eceeaf3-ab95-4000-913c-2254baf18b00\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hrbff"
Jul 17 09:34:07.869870 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.869851    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b11d398a-eb1b-4299-b1b1-85059ad5e910-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nql6d\" (UID: \"b11d398a-eb1b-4299-b1b1-85059ad5e910\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nql6d"
Jul 17 09:34:07.880154 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:34:07.880129    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5ts5r\" is forbidden: User \"system:node:ip-10-0-136-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-235.ec2.internal' and this object" podUID="100068db-bcc8-421c-81cc-736382c28819" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5ts5r"
Jul 17 09:34:07.880623 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.880604    2574 generic.go:356] "Generic (PLEG): container finished" podID="2619147d-4fbd-4688-8692-eaaab0b68f33" containerID="219795f7e23341dfbf4c6a0e1e5b9071dfac9dce07dde5e38f24288228e61456" exitCode=0
Jul 17 09:34:07.880703 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.880641    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx"
Jul 17 09:34:07.880703 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.880681    2574 scope.go:117] "RemoveContainer" containerID="219795f7e23341dfbf4c6a0e1e5b9071dfac9dce07dde5e38f24288228e61456"
Jul 17 09:34:07.882122 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.882090    2574 generic.go:356] "Generic (PLEG): container finished" podID="100068db-bcc8-421c-81cc-736382c28819" containerID="99b323499fb155147a641359f4e90c17a36e0e1a360405cc4c0037cb5f818b9d" exitCode=0
Jul 17 09:34:07.882214 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.882130    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5ts5r"
Jul 17 09:34:07.882928 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:34:07.882905    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rf5hx\" is forbidden: User \"system:node:ip-10-0-136-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-235.ec2.internal' and this object" podUID="2619147d-4fbd-4688-8692-eaaab0b68f33" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx"
Jul 17 09:34:07.885417 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:34:07.885389    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5ts5r\" is forbidden: User \"system:node:ip-10-0-136-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-235.ec2.internal' and this object" podUID="100068db-bcc8-421c-81cc-736382c28819" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5ts5r"
Jul 17 09:34:07.888148 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:34:07.888119    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rf5hx\" is forbidden: User \"system:node:ip-10-0-136-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-235.ec2.internal' and this object" podUID="2619147d-4fbd-4688-8692-eaaab0b68f33" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx"
Jul 17 09:34:07.889588 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.889563    2574 scope.go:117] "RemoveContainer" containerID="219795f7e23341dfbf4c6a0e1e5b9071dfac9dce07dde5e38f24288228e61456"
Jul 17 09:34:07.889736 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.889713    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm9st\" (UniqueName: \"kubernetes.io/projected/7eceeaf3-ab95-4000-913c-2254baf18b00-kube-api-access-mm9st\") pod \"limitador-operator-controller-manager-859d7b78bd-hrbff\" (UID: \"7eceeaf3-ab95-4000-913c-2254baf18b00\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hrbff"
Jul 17 09:34:07.889949 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.889931    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdx8r\" (UniqueName: \"kubernetes.io/projected/b11d398a-eb1b-4299-b1b1-85059ad5e910-kube-api-access-pdx8r\") pod \"kuadrant-operator-controller-manager-8684c9ff49-nql6d\" (UID: \"b11d398a-eb1b-4299-b1b1-85059ad5e910\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nql6d"
Jul 17 09:34:07.890041 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:34:07.889939    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"219795f7e23341dfbf4c6a0e1e5b9071dfac9dce07dde5e38f24288228e61456\": container with ID starting with 219795f7e23341dfbf4c6a0e1e5b9071dfac9dce07dde5e38f24288228e61456 not found: ID does not exist" containerID="219795f7e23341dfbf4c6a0e1e5b9071dfac9dce07dde5e38f24288228e61456"
Jul 17 09:34:07.890041 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.889974    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"219795f7e23341dfbf4c6a0e1e5b9071dfac9dce07dde5e38f24288228e61456"} err="failed to get container status \"219795f7e23341dfbf4c6a0e1e5b9071dfac9dce07dde5e38f24288228e61456\": rpc error: code = NotFound desc = could not find container \"219795f7e23341dfbf4c6a0e1e5b9071dfac9dce07dde5e38f24288228e61456\": container with ID starting with 219795f7e23341dfbf4c6a0e1e5b9071dfac9dce07dde5e38f24288228e61456 not found: ID does not exist"
Jul 17 09:34:07.890041 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.889996    2574 scope.go:117] "RemoveContainer" containerID="99b323499fb155147a641359f4e90c17a36e0e1a360405cc4c0037cb5f818b9d"
Jul 17 09:34:07.890935 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:34:07.890910    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5ts5r\" is forbidden: User \"system:node:ip-10-0-136-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-235.ec2.internal' and this object" podUID="100068db-bcc8-421c-81cc-736382c28819" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5ts5r"
Jul 17 09:34:07.897966 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.897949    2574 scope.go:117] "RemoveContainer" containerID="99b323499fb155147a641359f4e90c17a36e0e1a360405cc4c0037cb5f818b9d"
Jul 17 09:34:07.898226 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:34:07.898205    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99b323499fb155147a641359f4e90c17a36e0e1a360405cc4c0037cb5f818b9d\": container with ID starting with 99b323499fb155147a641359f4e90c17a36e0e1a360405cc4c0037cb5f818b9d not found: ID does not exist" containerID="99b323499fb155147a641359f4e90c17a36e0e1a360405cc4c0037cb5f818b9d"
Jul 17 09:34:07.898303 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.898231    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99b323499fb155147a641359f4e90c17a36e0e1a360405cc4c0037cb5f818b9d"} err="failed to get container status \"99b323499fb155147a641359f4e90c17a36e0e1a360405cc4c0037cb5f818b9d\": rpc error: code = NotFound desc = could not find container \"99b323499fb155147a641359f4e90c17a36e0e1a360405cc4c0037cb5f818b9d\": container with ID starting with 99b323499fb155147a641359f4e90c17a36e0e1a360405cc4c0037cb5f818b9d not found: ID does not exist"
Jul 17 09:34:07.970487 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.970461    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2619147d-4fbd-4688-8692-eaaab0b68f33-extensions-socket-volume\") pod \"2619147d-4fbd-4688-8692-eaaab0b68f33\" (UID: \"2619147d-4fbd-4688-8692-eaaab0b68f33\") "
Jul 17 09:34:07.970602 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.970501    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rt5r2\" (UniqueName: \"kubernetes.io/projected/2619147d-4fbd-4688-8692-eaaab0b68f33-kube-api-access-rt5r2\") pod \"2619147d-4fbd-4688-8692-eaaab0b68f33\" (UID: \"2619147d-4fbd-4688-8692-eaaab0b68f33\") "
Jul 17 09:34:07.970602 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.970520    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gfb49\" (UniqueName: \"kubernetes.io/projected/100068db-bcc8-421c-81cc-736382c28819-kube-api-access-gfb49\") pod \"100068db-bcc8-421c-81cc-736382c28819\" (UID: \"100068db-bcc8-421c-81cc-736382c28819\") "
Jul 17 09:34:07.970951 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.970916    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2619147d-4fbd-4688-8692-eaaab0b68f33-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2619147d-4fbd-4688-8692-eaaab0b68f33" (UID: "2619147d-4fbd-4688-8692-eaaab0b68f33"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:34:07.972523 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.972502    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2619147d-4fbd-4688-8692-eaaab0b68f33-kube-api-access-rt5r2" (OuterVolumeSpecName: "kube-api-access-rt5r2") pod "2619147d-4fbd-4688-8692-eaaab0b68f33" (UID: "2619147d-4fbd-4688-8692-eaaab0b68f33"). InnerVolumeSpecName "kube-api-access-rt5r2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:34:07.972606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:07.972557    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/100068db-bcc8-421c-81cc-736382c28819-kube-api-access-gfb49" (OuterVolumeSpecName: "kube-api-access-gfb49") pod "100068db-bcc8-421c-81cc-736382c28819" (UID: "100068db-bcc8-421c-81cc-736382c28819"). InnerVolumeSpecName "kube-api-access-gfb49". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:34:08.020533 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:08.020510    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nql6d"
Jul 17 09:34:08.027406 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:08.027392    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hrbff"
Jul 17 09:34:08.072727 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:08.072699    2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2619147d-4fbd-4688-8692-eaaab0b68f33-extensions-socket-volume\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:34:08.072822 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:08.072734    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rt5r2\" (UniqueName: \"kubernetes.io/projected/2619147d-4fbd-4688-8692-eaaab0b68f33-kube-api-access-rt5r2\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:34:08.072822 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:08.072754    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfb49\" (UniqueName: \"kubernetes.io/projected/100068db-bcc8-421c-81cc-736382c28819-kube-api-access-gfb49\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:34:08.170192 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:08.170150    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nql6d"]
Jul 17 09:34:08.174999 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:34:08.174973    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb11d398a_eb1b_4299_b1b1_85059ad5e910.slice/crio-e8f8d619b692d87b6e4230f9db06a0c3e7f95ef550b26090d6d3b68763741ff7 WatchSource:0}: Error finding container e8f8d619b692d87b6e4230f9db06a0c3e7f95ef550b26090d6d3b68763741ff7: Status 404 returned error can't find the container with id e8f8d619b692d87b6e4230f9db06a0c3e7f95ef550b26090d6d3b68763741ff7
Jul 17 09:34:08.184391 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:08.184368    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hrbff"]
Jul 17 09:34:08.185928 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:34:08.185901    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eceeaf3_ab95_4000_913c_2254baf18b00.slice/crio-225f39f9cd9a34297a9b6d19dfbe37a4c23bf3c760212dffcc404cf1fbb6ec6d WatchSource:0}: Error finding container 225f39f9cd9a34297a9b6d19dfbe37a4c23bf3c760212dffcc404cf1fbb6ec6d: Status 404 returned error can't find the container with id 225f39f9cd9a34297a9b6d19dfbe37a4c23bf3c760212dffcc404cf1fbb6ec6d
Jul 17 09:34:08.195308 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:34:08.195274    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rf5hx\" is forbidden: User \"system:node:ip-10-0-136-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-235.ec2.internal' and this object" podUID="2619147d-4fbd-4688-8692-eaaab0b68f33" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx"
Jul 17 09:34:08.197406 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:34:08.197380    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5ts5r\" is forbidden: User \"system:node:ip-10-0-136-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-235.ec2.internal' and this object" podUID="100068db-bcc8-421c-81cc-736382c28819" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5ts5r"
Jul 17 09:34:08.199439 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:34:08.199414    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5ts5r\" is forbidden: User \"system:node:ip-10-0-136-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-235.ec2.internal' and this object" podUID="100068db-bcc8-421c-81cc-736382c28819" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5ts5r"
Jul 17 09:34:08.201453 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:34:08.201424    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rf5hx\" is forbidden: User \"system:node:ip-10-0-136-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-235.ec2.internal' and this object" podUID="2619147d-4fbd-4688-8692-eaaab0b68f33" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx"
Jul 17 09:34:08.509959 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:08.509926    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lmklg"]
Jul 17 09:34:08.513559 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:08.513537    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lmklg"
Jul 17 09:34:08.516686 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:34:08.516658    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rf5hx\" is forbidden: User \"system:node:ip-10-0-136-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-235.ec2.internal' and this object" podUID="2619147d-4fbd-4688-8692-eaaab0b68f33" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx"
Jul 17 09:34:08.538644 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:08.538621    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lmklg"]
Jul 17 09:34:08.579209 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:34:08.579175    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5ts5r\" is forbidden: User \"system:node:ip-10-0-136-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-235.ec2.internal' and this object" podUID="100068db-bcc8-421c-81cc-736382c28819" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5ts5r"
Jul 17 09:34:08.678376 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:08.678345    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9e42161-af91-4958-bebe-2a87ff299a1a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-lmklg\" (UID: \"f9e42161-af91-4958-bebe-2a87ff299a1a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lmklg"
Jul 17 09:34:08.678518 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:08.678394    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpdwf\" (UniqueName: \"kubernetes.io/projected/f9e42161-af91-4958-bebe-2a87ff299a1a-kube-api-access-qpdwf\") pod \"kuadrant-operator-controller-manager-86878c769f-lmklg\" (UID: \"f9e42161-af91-4958-bebe-2a87ff299a1a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lmklg"
Jul 17 09:34:08.779887 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:08.779802    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9e42161-af91-4958-bebe-2a87ff299a1a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-lmklg\" (UID: \"f9e42161-af91-4958-bebe-2a87ff299a1a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lmklg"
Jul 17 09:34:08.779887 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:08.779855    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpdwf\" (UniqueName: \"kubernetes.io/projected/f9e42161-af91-4958-bebe-2a87ff299a1a-kube-api-access-qpdwf\") pod \"kuadrant-operator-controller-manager-86878c769f-lmklg\" (UID: \"f9e42161-af91-4958-bebe-2a87ff299a1a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lmklg"
Jul 17 09:34:08.780165 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:08.780147    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9e42161-af91-4958-bebe-2a87ff299a1a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-lmklg\" (UID: \"f9e42161-af91-4958-bebe-2a87ff299a1a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lmklg"
Jul 17 09:34:08.790024 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:08.789993    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpdwf\" (UniqueName: \"kubernetes.io/projected/f9e42161-af91-4958-bebe-2a87ff299a1a-kube-api-access-qpdwf\") pod \"kuadrant-operator-controller-manager-86878c769f-lmklg\" (UID: \"f9e42161-af91-4958-bebe-2a87ff299a1a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lmklg"
Jul 17 09:34:08.823718 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:08.823692    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lmklg"
Jul 17 09:34:08.895110 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:34:08.895071    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-rf5hx\" is forbidden: User \"system:node:ip-10-0-136-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-235.ec2.internal' and this object" podUID="2619147d-4fbd-4688-8692-eaaab0b68f33" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rf5hx"
Jul 17 09:34:08.897445 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:34:08.897414    2574 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5ts5r\" is forbidden: User \"system:node:ip-10-0-136-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-235.ec2.internal' and this object" podUID="100068db-bcc8-421c-81cc-736382c28819" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5ts5r"
Jul 17 09:34:08.897827 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:08.897808    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="100068db-bcc8-421c-81cc-736382c28819" path="/var/lib/kubelet/pods/100068db-bcc8-421c-81cc-736382c28819/volumes"
Jul 17 09:34:08.898153 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:08.898141    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2619147d-4fbd-4688-8692-eaaab0b68f33" path="/var/lib/kubelet/pods/2619147d-4fbd-4688-8692-eaaab0b68f33/volumes"
Jul 17 09:34:08.898444 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:08.898427    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hrbff" event={"ID":"7eceeaf3-ab95-4000-913c-2254baf18b00","Type":"ContainerStarted","Data":"a81502bf1d0bef8670c6e967cbe76f632afab41d46a0a69be2b23263ce8dc8b9"}
Jul 17 09:34:08.898486 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:08.898472    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hrbff"
Jul 17 09:34:08.898524 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:08.898489    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hrbff" event={"ID":"7eceeaf3-ab95-4000-913c-2254baf18b00","Type":"ContainerStarted","Data":"225f39f9cd9a34297a9b6d19dfbe37a4c23bf3c760212dffcc404cf1fbb6ec6d"}
Jul 17 09:34:08.898524 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:08.898504    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nql6d"
Jul 17 09:34:08.898524 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:08.898518    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nql6d" event={"ID":"b11d398a-eb1b-4299-b1b1-85059ad5e910","Type":"ContainerStarted","Data":"e899829089b0e519c0dba932143426f52a3d6bfc24a8b77a8b15f71f5b0f55cd"}
Jul 17 09:34:08.898608 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:08.898534    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nql6d" event={"ID":"b11d398a-eb1b-4299-b1b1-85059ad5e910","Type":"ContainerStarted","Data":"e8f8d619b692d87b6e4230f9db06a0c3e7f95ef550b26090d6d3b68763741ff7"}
Jul 17 09:34:08.916279 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:08.916213    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hrbff" podStartSLOduration=1.916196265 podStartE2EDuration="1.916196265s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:34:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:34:08.914137802 +0000 UTC m=+446.496776202" watchObservedRunningTime="2026-07-17 09:34:08.916196265 +0000 UTC m=+446.498834644"
Jul 17 09:34:08.932387 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:08.932340    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nql6d" podStartSLOduration=1.9323235890000001 podStartE2EDuration="1.932323589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:34:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:34:08.931860981 +0000 UTC m=+446.514499358" watchObservedRunningTime="2026-07-17 09:34:08.932323589 +0000 UTC m=+446.514961966"
Jul 17 09:34:08.971702 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:08.971681    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lmklg"]
Jul 17 09:34:08.973915 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:34:08.973886    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9e42161_af91_4958_bebe_2a87ff299a1a.slice/crio-1a585ba25d03ecefdf49c929b37a1b5fb6044ec303ec20608fc2953882634333 WatchSource:0}: Error finding container 1a585ba25d03ecefdf49c929b37a1b5fb6044ec303ec20608fc2953882634333: Status 404 returned error can't find the container with id 1a585ba25d03ecefdf49c929b37a1b5fb6044ec303ec20608fc2953882634333
Jul 17 09:34:09.900817 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:09.900778    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lmklg" event={"ID":"f9e42161-af91-4958-bebe-2a87ff299a1a","Type":"ContainerStarted","Data":"4e5b6d9a8bb38d6f5fe068e7a0116fd4f715cc4a9cec30fdf3204db1ecfd3897"}
Jul 17 09:34:09.901355 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:09.900869    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lmklg" event={"ID":"f9e42161-af91-4958-bebe-2a87ff299a1a","Type":"ContainerStarted","Data":"1a585ba25d03ecefdf49c929b37a1b5fb6044ec303ec20608fc2953882634333"}
Jul 17 09:34:09.901355 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:09.901100    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lmklg"
Jul 17 09:34:09.929321 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:09.929280    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lmklg" podStartSLOduration=1.929261435 podStartE2EDuration="1.929261435s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:34:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:34:09.927061657 +0000 UTC m=+447.509700108" watchObservedRunningTime="2026-07-17 09:34:09.929261435 +0000 UTC m=+447.511899813"
Jul 17 09:34:19.902880 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:19.902852    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-hrbff"
Jul 17 09:34:20.907896 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:20.907869    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nql6d"
Jul 17 09:34:20.908315 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:20.908033    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lmklg"
Jul 17 09:34:20.989213 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:20.989179    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nql6d"]
Jul 17 09:34:20.989437 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:20.989407    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nql6d" podUID="b11d398a-eb1b-4299-b1b1-85059ad5e910" containerName="manager" containerID="cri-o://e899829089b0e519c0dba932143426f52a3d6bfc24a8b77a8b15f71f5b0f55cd" gracePeriod=10
Jul 17 09:34:21.234960 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:21.234939    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nql6d"
Jul 17 09:34:21.373327 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:21.373299    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b11d398a-eb1b-4299-b1b1-85059ad5e910-extensions-socket-volume\") pod \"b11d398a-eb1b-4299-b1b1-85059ad5e910\" (UID: \"b11d398a-eb1b-4299-b1b1-85059ad5e910\") "
Jul 17 09:34:21.373459 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:21.373396    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdx8r\" (UniqueName: \"kubernetes.io/projected/b11d398a-eb1b-4299-b1b1-85059ad5e910-kube-api-access-pdx8r\") pod \"b11d398a-eb1b-4299-b1b1-85059ad5e910\" (UID: \"b11d398a-eb1b-4299-b1b1-85059ad5e910\") "
Jul 17 09:34:21.373652 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:21.373620    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b11d398a-eb1b-4299-b1b1-85059ad5e910-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b11d398a-eb1b-4299-b1b1-85059ad5e910" (UID: "b11d398a-eb1b-4299-b1b1-85059ad5e910"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:34:21.375395 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:21.375375    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b11d398a-eb1b-4299-b1b1-85059ad5e910-kube-api-access-pdx8r" (OuterVolumeSpecName: "kube-api-access-pdx8r") pod "b11d398a-eb1b-4299-b1b1-85059ad5e910" (UID: "b11d398a-eb1b-4299-b1b1-85059ad5e910"). InnerVolumeSpecName "kube-api-access-pdx8r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:34:21.473958 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:21.473904    2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b11d398a-eb1b-4299-b1b1-85059ad5e910-extensions-socket-volume\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:34:21.473958 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:21.473923    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdx8r\" (UniqueName: \"kubernetes.io/projected/b11d398a-eb1b-4299-b1b1-85059ad5e910-kube-api-access-pdx8r\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:34:21.962020 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:21.961994    2574 generic.go:356] "Generic (PLEG): container finished" podID="b11d398a-eb1b-4299-b1b1-85059ad5e910" containerID="e899829089b0e519c0dba932143426f52a3d6bfc24a8b77a8b15f71f5b0f55cd" exitCode=0
Jul 17 09:34:21.962375 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:21.962039    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nql6d"
Jul 17 09:34:21.962375 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:21.962077    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nql6d" event={"ID":"b11d398a-eb1b-4299-b1b1-85059ad5e910","Type":"ContainerDied","Data":"e899829089b0e519c0dba932143426f52a3d6bfc24a8b77a8b15f71f5b0f55cd"}
Jul 17 09:34:21.962375 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:21.962115    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nql6d" event={"ID":"b11d398a-eb1b-4299-b1b1-85059ad5e910","Type":"ContainerDied","Data":"e8f8d619b692d87b6e4230f9db06a0c3e7f95ef550b26090d6d3b68763741ff7"}
Jul 17 09:34:21.962375 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:21.962135    2574 scope.go:117] "RemoveContainer" containerID="e899829089b0e519c0dba932143426f52a3d6bfc24a8b77a8b15f71f5b0f55cd"
Jul 17 09:34:21.972800 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:21.972768    2574 scope.go:117] "RemoveContainer" containerID="e899829089b0e519c0dba932143426f52a3d6bfc24a8b77a8b15f71f5b0f55cd"
Jul 17 09:34:21.973053 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:34:21.973036    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e899829089b0e519c0dba932143426f52a3d6bfc24a8b77a8b15f71f5b0f55cd\": container with ID starting with e899829089b0e519c0dba932143426f52a3d6bfc24a8b77a8b15f71f5b0f55cd not found: ID does not exist" containerID="e899829089b0e519c0dba932143426f52a3d6bfc24a8b77a8b15f71f5b0f55cd"
Jul 17 09:34:21.973105 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:21.973058    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e899829089b0e519c0dba932143426f52a3d6bfc24a8b77a8b15f71f5b0f55cd"} err="failed to get container status \"e899829089b0e519c0dba932143426f52a3d6bfc24a8b77a8b15f71f5b0f55cd\": rpc error: code = NotFound desc = could not find container \"e899829089b0e519c0dba932143426f52a3d6bfc24a8b77a8b15f71f5b0f55cd\": container with ID starting with e899829089b0e519c0dba932143426f52a3d6bfc24a8b77a8b15f71f5b0f55cd not found: ID does not exist"
Jul 17 09:34:21.988762 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:21.988735    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nql6d"]
Jul 17 09:34:21.993489 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:21.993466    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-nql6d"]
Jul 17 09:34:22.897475 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:22.897448    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b11d398a-eb1b-4299-b1b1-85059ad5e910" path="/var/lib/kubelet/pods/b11d398a-eb1b-4299-b1b1-85059ad5e910/volumes"
Jul 17 09:34:37.280521 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.280255    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"]
Jul 17 09:34:37.280991 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.280739    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b11d398a-eb1b-4299-b1b1-85059ad5e910" containerName="manager"
Jul 17 09:34:37.280991 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.280754    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b11d398a-eb1b-4299-b1b1-85059ad5e910" containerName="manager"
Jul 17 09:34:37.280991 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.280824    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b11d398a-eb1b-4299-b1b1-85059ad5e910" containerName="manager"
Jul 17 09:34:37.283305 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.283284    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.287139 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.287073    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-6wqkc\""
Jul 17 09:34:37.287851 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.287830    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 17 09:34:37.299299 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.299231    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"]
Jul 17 09:34:37.396284 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.396233    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aff066d2-5135-49e2-b041-39424acf8fb6-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-x5rwn\" (UID: \"aff066d2-5135-49e2-b041-39424acf8fb6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.396381 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.396286    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aff066d2-5135-49e2-b041-39424acf8fb6-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-x5rwn\" (UID: \"aff066d2-5135-49e2-b041-39424acf8fb6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.396381 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.396304    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aff066d2-5135-49e2-b041-39424acf8fb6-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-x5rwn\" (UID: \"aff066d2-5135-49e2-b041-39424acf8fb6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.396381 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.396343    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aff066d2-5135-49e2-b041-39424acf8fb6-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-x5rwn\" (UID: \"aff066d2-5135-49e2-b041-39424acf8fb6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.396381 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.396378    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aff066d2-5135-49e2-b041-39424acf8fb6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-x5rwn\" (UID: \"aff066d2-5135-49e2-b041-39424acf8fb6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.396536 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.396394    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aff066d2-5135-49e2-b041-39424acf8fb6-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-x5rwn\" (UID: \"aff066d2-5135-49e2-b041-39424acf8fb6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.396536 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.396443    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4mmt\" (UniqueName: \"kubernetes.io/projected/aff066d2-5135-49e2-b041-39424acf8fb6-kube-api-access-c4mmt\") pod \"maas-default-gateway-openshift-default-56654dcf7-x5rwn\" (UID: \"aff066d2-5135-49e2-b041-39424acf8fb6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.396536 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.396470    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aff066d2-5135-49e2-b041-39424acf8fb6-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-x5rwn\" (UID: \"aff066d2-5135-49e2-b041-39424acf8fb6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.396536 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.396488    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aff066d2-5135-49e2-b041-39424acf8fb6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-x5rwn\" (UID: \"aff066d2-5135-49e2-b041-39424acf8fb6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.497204 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.497178    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aff066d2-5135-49e2-b041-39424acf8fb6-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-x5rwn\" (UID: \"aff066d2-5135-49e2-b041-39424acf8fb6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.497204 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.497204    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aff066d2-5135-49e2-b041-39424acf8fb6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-x5rwn\" (UID: \"aff066d2-5135-49e2-b041-39424acf8fb6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.497381 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.497295    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aff066d2-5135-49e2-b041-39424acf8fb6-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-x5rwn\" (UID: \"aff066d2-5135-49e2-b041-39424acf8fb6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.497381 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.497312    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aff066d2-5135-49e2-b041-39424acf8fb6-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-x5rwn\" (UID: \"aff066d2-5135-49e2-b041-39424acf8fb6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.497381 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.497330    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aff066d2-5135-49e2-b041-39424acf8fb6-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-x5rwn\" (UID: \"aff066d2-5135-49e2-b041-39424acf8fb6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.497381 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.497352    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aff066d2-5135-49e2-b041-39424acf8fb6-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-x5rwn\" (UID: \"aff066d2-5135-49e2-b041-39424acf8fb6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.497576 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.497399    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aff066d2-5135-49e2-b041-39424acf8fb6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-x5rwn\" (UID: \"aff066d2-5135-49e2-b041-39424acf8fb6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.497576 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.497435    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aff066d2-5135-49e2-b041-39424acf8fb6-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-x5rwn\" (UID: \"aff066d2-5135-49e2-b041-39424acf8fb6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.497576 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.497470    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4mmt\" (UniqueName: \"kubernetes.io/projected/aff066d2-5135-49e2-b041-39424acf8fb6-kube-api-access-c4mmt\") pod \"maas-default-gateway-openshift-default-56654dcf7-x5rwn\" (UID: \"aff066d2-5135-49e2-b041-39424acf8fb6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.497763 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.497655    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aff066d2-5135-49e2-b041-39424acf8fb6-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-x5rwn\" (UID: \"aff066d2-5135-49e2-b041-39424acf8fb6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.497923 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.497892    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aff066d2-5135-49e2-b041-39424acf8fb6-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-x5rwn\" (UID: \"aff066d2-5135-49e2-b041-39424acf8fb6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.498056 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.498029    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aff066d2-5135-49e2-b041-39424acf8fb6-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-x5rwn\" (UID: \"aff066d2-5135-49e2-b041-39424acf8fb6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.498125 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.498085    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aff066d2-5135-49e2-b041-39424acf8fb6-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-x5rwn\" (UID: \"aff066d2-5135-49e2-b041-39424acf8fb6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.498428 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.498404    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aff066d2-5135-49e2-b041-39424acf8fb6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-x5rwn\" (UID: \"aff066d2-5135-49e2-b041-39424acf8fb6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.499737 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.499716    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aff066d2-5135-49e2-b041-39424acf8fb6-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-x5rwn\" (UID: \"aff066d2-5135-49e2-b041-39424acf8fb6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.500011 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.499984    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aff066d2-5135-49e2-b041-39424acf8fb6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-x5rwn\" (UID: \"aff066d2-5135-49e2-b041-39424acf8fb6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.506442 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.506422    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aff066d2-5135-49e2-b041-39424acf8fb6-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-x5rwn\" (UID: \"aff066d2-5135-49e2-b041-39424acf8fb6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.506657 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.506641    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4mmt\" (UniqueName: \"kubernetes.io/projected/aff066d2-5135-49e2-b041-39424acf8fb6-kube-api-access-c4mmt\") pod \"maas-default-gateway-openshift-default-56654dcf7-x5rwn\" (UID: \"aff066d2-5135-49e2-b041-39424acf8fb6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.600852 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.600800    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:37.737899 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:37.735634    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"]
Jul 17 09:34:37.737899 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:34:37.737850    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaff066d2_5135_49e2_b041_39424acf8fb6.slice/crio-c61cae2cd65bd3ffca878094bcc4baa565572e053b9fd737381ee7a191f7d04c WatchSource:0}: Error finding container c61cae2cd65bd3ffca878094bcc4baa565572e053b9fd737381ee7a191f7d04c: Status 404 returned error can't find the container with id c61cae2cd65bd3ffca878094bcc4baa565572e053b9fd737381ee7a191f7d04c
Jul 17 09:34:38.039681 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:38.039654    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn" event={"ID":"aff066d2-5135-49e2-b041-39424acf8fb6","Type":"ContainerStarted","Data":"c61cae2cd65bd3ffca878094bcc4baa565572e053b9fd737381ee7a191f7d04c"}
Jul 17 09:34:40.124359 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:40.124323    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 09:34:40.124590 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:40.124398    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 09:34:40.124590 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:40.124431    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 09:34:41.057003 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:41.056967    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn" event={"ID":"aff066d2-5135-49e2-b041-39424acf8fb6","Type":"ContainerStarted","Data":"dc17cca57d50bcf584724bd24d7d8ea296bef0a3a26b9326972563dedab1b177"}
Jul 17 09:34:41.081015 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:41.080968    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn" podStartSLOduration=1.698797254 podStartE2EDuration="4.080954631s" totalImagesPullingTime="2.382157377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:34:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:34:41.079657607 +0000 UTC m=+478.662295985" watchObservedRunningTime="2026-07-17 09:34:41.080954631 +0000 UTC m=+478.663593007"
Jul 17 09:34:41.601609 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:41.601580    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:41.606167 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:41.606146    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:42.062763 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:42.062735    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:42.063749 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:42.063733    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:34:42.098636 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:42.098601    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jdkrl"]
Jul 17 09:34:42.103056 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:42.103030    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jdkrl"
Jul 17 09:34:42.106766 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:42.106730    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-k2wpb\""
Jul 17 09:34:42.106999 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:42.106981    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 17 09:34:42.107903 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:42.107879    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jdkrl"]
Jul 17 09:34:42.137383 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:42.137355    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktkb2\" (UniqueName: \"kubernetes.io/projected/f56e6dd9-b52f-45d9-abcb-9eb993d82495-kube-api-access-ktkb2\") pod \"limitador-limitador-64c94fd64f-jdkrl\" (UID: \"f56e6dd9-b52f-45d9-abcb-9eb993d82495\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jdkrl"
Jul 17 09:34:42.137634 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:42.137594    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f56e6dd9-b52f-45d9-abcb-9eb993d82495-config-file\") pod \"limitador-limitador-64c94fd64f-jdkrl\" (UID: \"f56e6dd9-b52f-45d9-abcb-9eb993d82495\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jdkrl"
Jul 17 09:34:42.208088 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:42.208061    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jdkrl"]
Jul 17 09:34:42.238673 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:42.238648    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f56e6dd9-b52f-45d9-abcb-9eb993d82495-config-file\") pod \"limitador-limitador-64c94fd64f-jdkrl\" (UID: \"f56e6dd9-b52f-45d9-abcb-9eb993d82495\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jdkrl"
Jul 17 09:34:42.238799 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:42.238691    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktkb2\" (UniqueName: \"kubernetes.io/projected/f56e6dd9-b52f-45d9-abcb-9eb993d82495-kube-api-access-ktkb2\") pod \"limitador-limitador-64c94fd64f-jdkrl\" (UID: \"f56e6dd9-b52f-45d9-abcb-9eb993d82495\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jdkrl"
Jul 17 09:34:42.239278 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:42.239228    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f56e6dd9-b52f-45d9-abcb-9eb993d82495-config-file\") pod \"limitador-limitador-64c94fd64f-jdkrl\" (UID: \"f56e6dd9-b52f-45d9-abcb-9eb993d82495\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jdkrl"
Jul 17 09:34:42.257223 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:42.257198    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktkb2\" (UniqueName: \"kubernetes.io/projected/f56e6dd9-b52f-45d9-abcb-9eb993d82495-kube-api-access-ktkb2\") pod \"limitador-limitador-64c94fd64f-jdkrl\" (UID: \"f56e6dd9-b52f-45d9-abcb-9eb993d82495\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jdkrl"
Jul 17 09:34:42.418075 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:42.418050    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jdkrl"
Jul 17 09:34:42.565415 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:42.565378    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jdkrl"]
Jul 17 09:34:42.569276 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:34:42.569222    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf56e6dd9_b52f_45d9_abcb_9eb993d82495.slice/crio-ba3da789fb850018ce59e54a98eb64005e8a4cd69c84fb3e4a59e9e32a2bbf28 WatchSource:0}: Error finding container ba3da789fb850018ce59e54a98eb64005e8a4cd69c84fb3e4a59e9e32a2bbf28: Status 404 returned error can't find the container with id ba3da789fb850018ce59e54a98eb64005e8a4cd69c84fb3e4a59e9e32a2bbf28
Jul 17 09:34:43.068122 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:43.068077    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jdkrl" event={"ID":"f56e6dd9-b52f-45d9-abcb-9eb993d82495","Type":"ContainerStarted","Data":"ba3da789fb850018ce59e54a98eb64005e8a4cd69c84fb3e4a59e9e32a2bbf28"}
Jul 17 09:34:46.087268 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:46.087216    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jdkrl" event={"ID":"f56e6dd9-b52f-45d9-abcb-9eb993d82495","Type":"ContainerStarted","Data":"4c4dfd2fabc51893eb7e7adbac16498996d869f8e35df5cbb1d3dcbc91d692f6"}
Jul 17 09:34:46.087639 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:46.087336    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-jdkrl"
Jul 17 09:34:46.103599 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:46.103550    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-jdkrl" podStartSLOduration=1.508958912 podStartE2EDuration="4.10353834s" totalImagesPullingTime="2.594579428s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:34:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:34:46.103414061 +0000 UTC m=+483.686052438" watchObservedRunningTime="2026-07-17 09:34:46.10353834 +0000 UTC m=+483.686176715"
Jul 17 09:34:57.095163 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:57.095128    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-jdkrl"
Jul 17 09:34:59.648259 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:59.648206    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jdkrl"]
Jul 17 09:34:59.648689 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:34:59.648396    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-jdkrl" podUID="f56e6dd9-b52f-45d9-abcb-9eb993d82495" containerName="limitador" containerID="cri-o://4c4dfd2fabc51893eb7e7adbac16498996d869f8e35df5cbb1d3dcbc91d692f6" gracePeriod=30
Jul 17 09:35:00.157763 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:00.157732    2574 generic.go:356] "Generic (PLEG): container finished" podID="f56e6dd9-b52f-45d9-abcb-9eb993d82495" containerID="4c4dfd2fabc51893eb7e7adbac16498996d869f8e35df5cbb1d3dcbc91d692f6" exitCode=0
Jul 17 09:35:00.157914 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:00.157769    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jdkrl" event={"ID":"f56e6dd9-b52f-45d9-abcb-9eb993d82495","Type":"ContainerDied","Data":"4c4dfd2fabc51893eb7e7adbac16498996d869f8e35df5cbb1d3dcbc91d692f6"}
Jul 17 09:35:00.192695 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:00.192667    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jdkrl"
Jul 17 09:35:00.290860 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:00.290769    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ktkb2\" (UniqueName: \"kubernetes.io/projected/f56e6dd9-b52f-45d9-abcb-9eb993d82495-kube-api-access-ktkb2\") pod \"f56e6dd9-b52f-45d9-abcb-9eb993d82495\" (UID: \"f56e6dd9-b52f-45d9-abcb-9eb993d82495\") "
Jul 17 09:35:00.290860 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:00.290812    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f56e6dd9-b52f-45d9-abcb-9eb993d82495-config-file\") pod \"f56e6dd9-b52f-45d9-abcb-9eb993d82495\" (UID: \"f56e6dd9-b52f-45d9-abcb-9eb993d82495\") "
Jul 17 09:35:00.291209 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:00.291187    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f56e6dd9-b52f-45d9-abcb-9eb993d82495-config-file" (OuterVolumeSpecName: "config-file") pod "f56e6dd9-b52f-45d9-abcb-9eb993d82495" (UID: "f56e6dd9-b52f-45d9-abcb-9eb993d82495"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:35:00.292952 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:00.292923    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f56e6dd9-b52f-45d9-abcb-9eb993d82495-kube-api-access-ktkb2" (OuterVolumeSpecName: "kube-api-access-ktkb2") pod "f56e6dd9-b52f-45d9-abcb-9eb993d82495" (UID: "f56e6dd9-b52f-45d9-abcb-9eb993d82495"). InnerVolumeSpecName "kube-api-access-ktkb2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:35:00.391810 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:00.391768    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktkb2\" (UniqueName: \"kubernetes.io/projected/f56e6dd9-b52f-45d9-abcb-9eb993d82495-kube-api-access-ktkb2\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:35:00.391810 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:00.391806    2574 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f56e6dd9-b52f-45d9-abcb-9eb993d82495-config-file\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:35:01.163979 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:01.163954    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jdkrl"
Jul 17 09:35:01.164434 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:01.163953    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jdkrl" event={"ID":"f56e6dd9-b52f-45d9-abcb-9eb993d82495","Type":"ContainerDied","Data":"ba3da789fb850018ce59e54a98eb64005e8a4cd69c84fb3e4a59e9e32a2bbf28"}
Jul 17 09:35:01.164434 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:01.164078    2574 scope.go:117] "RemoveContainer" containerID="4c4dfd2fabc51893eb7e7adbac16498996d869f8e35df5cbb1d3dcbc91d692f6"
Jul 17 09:35:01.181838 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:01.181815    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jdkrl"]
Jul 17 09:35:01.185579 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:01.185547    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jdkrl"]
Jul 17 09:35:02.897465 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:02.897437    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f56e6dd9-b52f-45d9-abcb-9eb993d82495" path="/var/lib/kubelet/pods/f56e6dd9-b52f-45d9-abcb-9eb993d82495/volumes"
Jul 17 09:35:03.802302 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:03.802234    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-lrl7q"]
Jul 17 09:35:03.802860 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:03.802845    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f56e6dd9-b52f-45d9-abcb-9eb993d82495" containerName="limitador"
Jul 17 09:35:03.802901 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:03.802865    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f56e6dd9-b52f-45d9-abcb-9eb993d82495" containerName="limitador"
Jul 17 09:35:03.802996 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:03.802985    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f56e6dd9-b52f-45d9-abcb-9eb993d82495" containerName="limitador"
Jul 17 09:35:03.806510 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:03.806487    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lrl7q"
Jul 17 09:35:03.809549 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:03.809456    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 17 09:35:03.809549 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:03.809519    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 17 09:35:03.809752 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:03.809520    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-zrqbp\""
Jul 17 09:35:03.810736 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:03.810716    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 17 09:35:03.813592 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:03.813569    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-lrl7q"]
Jul 17 09:35:03.921659 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:03.921620    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/645866b9-eee2-4b8c-b3cd-131e8feff4ae-data\") pod \"postgres-69fb4594cf-lrl7q\" (UID: \"645866b9-eee2-4b8c-b3cd-131e8feff4ae\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lrl7q"
Jul 17 09:35:03.922080 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:03.921787    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gk4vd\" (UniqueName: \"kubernetes.io/projected/645866b9-eee2-4b8c-b3cd-131e8feff4ae-kube-api-access-gk4vd\") pod \"postgres-69fb4594cf-lrl7q\" (UID: \"645866b9-eee2-4b8c-b3cd-131e8feff4ae\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lrl7q"
Jul 17 09:35:04.022642 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:04.022613    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk4vd\" (UniqueName: \"kubernetes.io/projected/645866b9-eee2-4b8c-b3cd-131e8feff4ae-kube-api-access-gk4vd\") pod \"postgres-69fb4594cf-lrl7q\" (UID: \"645866b9-eee2-4b8c-b3cd-131e8feff4ae\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lrl7q"
Jul 17 09:35:04.022784 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:04.022665    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/645866b9-eee2-4b8c-b3cd-131e8feff4ae-data\") pod \"postgres-69fb4594cf-lrl7q\" (UID: \"645866b9-eee2-4b8c-b3cd-131e8feff4ae\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lrl7q"
Jul 17 09:35:04.022971 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:04.022954    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/645866b9-eee2-4b8c-b3cd-131e8feff4ae-data\") pod \"postgres-69fb4594cf-lrl7q\" (UID: \"645866b9-eee2-4b8c-b3cd-131e8feff4ae\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lrl7q"
Jul 17 09:35:04.030615 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:04.030594    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gk4vd\" (UniqueName: \"kubernetes.io/projected/645866b9-eee2-4b8c-b3cd-131e8feff4ae-kube-api-access-gk4vd\") pod \"postgres-69fb4594cf-lrl7q\" (UID: \"645866b9-eee2-4b8c-b3cd-131e8feff4ae\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lrl7q"
Jul 17 09:35:04.118888 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:04.118825    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lrl7q"
Jul 17 09:35:04.246863 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:04.246835    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-lrl7q"]
Jul 17 09:35:04.248631 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:35:04.248605    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod645866b9_eee2_4b8c_b3cd_131e8feff4ae.slice/crio-6a567c10b425f32d72586d31bddbf21733320690a5766f3eda195d6d7335f061 WatchSource:0}: Error finding container 6a567c10b425f32d72586d31bddbf21733320690a5766f3eda195d6d7335f061: Status 404 returned error can't find the container with id 6a567c10b425f32d72586d31bddbf21733320690a5766f3eda195d6d7335f061
Jul 17 09:35:05.189394 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:05.189356    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lrl7q" event={"ID":"645866b9-eee2-4b8c-b3cd-131e8feff4ae","Type":"ContainerStarted","Data":"6a567c10b425f32d72586d31bddbf21733320690a5766f3eda195d6d7335f061"}
Jul 17 09:35:09.212549 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:09.212510    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lrl7q" event={"ID":"645866b9-eee2-4b8c-b3cd-131e8feff4ae","Type":"ContainerStarted","Data":"16bcbcc9150d1aaa653fec3a2eb6c992750b6310992b5bffb131c422e3ca6a5d"}
Jul 17 09:35:09.212935 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:09.212642    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lrl7q"
Jul 17 09:35:09.232120 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:09.232068    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lrl7q" podStartSLOduration=1.78575211 podStartE2EDuration="6.232050124s" totalImagesPullingTime="4.446298014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:35:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:35:09.227539752 +0000 UTC m=+506.810178154" watchObservedRunningTime="2026-07-17 09:35:09.232050124 +0000 UTC m=+506.814688501"
Jul 17 09:35:15.247747 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:15.247714    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lrl7q"
Jul 17 09:35:22.489505 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:22.489467    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7cf56599d4-fz8k4"]
Jul 17 09:35:22.498345 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:22.498321    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cf56599d4-fz8k4"
Jul 17 09:35:22.502739 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:22.502710    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 17 09:35:22.502739 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:22.502736    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jc8mw\""
Jul 17 09:35:22.502966 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:22.502722    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 17 09:35:22.507540 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:22.507516    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7cf56599d4-fz8k4"]
Jul 17 09:35:22.592889 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:22.592863    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/37b1a86a-13db-4609-923c-fbaf9cd4f8da-webhook-cert\") pod \"maas-controller-7cf56599d4-fz8k4\" (UID: \"37b1a86a-13db-4609-923c-fbaf9cd4f8da\") " pod="opendatahub/maas-controller-7cf56599d4-fz8k4"
Jul 17 09:35:22.592889 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:22.592891    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lmp8\" (UniqueName: \"kubernetes.io/projected/37b1a86a-13db-4609-923c-fbaf9cd4f8da-kube-api-access-4lmp8\") pod \"maas-controller-7cf56599d4-fz8k4\" (UID: \"37b1a86a-13db-4609-923c-fbaf9cd4f8da\") " pod="opendatahub/maas-controller-7cf56599d4-fz8k4"
Jul 17 09:35:22.693723 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:22.693684    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/37b1a86a-13db-4609-923c-fbaf9cd4f8da-webhook-cert\") pod \"maas-controller-7cf56599d4-fz8k4\" (UID: \"37b1a86a-13db-4609-923c-fbaf9cd4f8da\") " pod="opendatahub/maas-controller-7cf56599d4-fz8k4"
Jul 17 09:35:22.693882 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:22.693725    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lmp8\" (UniqueName: \"kubernetes.io/projected/37b1a86a-13db-4609-923c-fbaf9cd4f8da-kube-api-access-4lmp8\") pod \"maas-controller-7cf56599d4-fz8k4\" (UID: \"37b1a86a-13db-4609-923c-fbaf9cd4f8da\") " pod="opendatahub/maas-controller-7cf56599d4-fz8k4"
Jul 17 09:35:22.696161 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:22.696137    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/37b1a86a-13db-4609-923c-fbaf9cd4f8da-webhook-cert\") pod \"maas-controller-7cf56599d4-fz8k4\" (UID: \"37b1a86a-13db-4609-923c-fbaf9cd4f8da\") " pod="opendatahub/maas-controller-7cf56599d4-fz8k4"
Jul 17 09:35:22.702715 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:22.702682    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lmp8\" (UniqueName: \"kubernetes.io/projected/37b1a86a-13db-4609-923c-fbaf9cd4f8da-kube-api-access-4lmp8\") pod \"maas-controller-7cf56599d4-fz8k4\" (UID: \"37b1a86a-13db-4609-923c-fbaf9cd4f8da\") " pod="opendatahub/maas-controller-7cf56599d4-fz8k4"
Jul 17 09:35:22.810062 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:22.810000    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cf56599d4-fz8k4"
Jul 17 09:35:22.943207 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:22.943185    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7cf56599d4-fz8k4"]
Jul 17 09:35:22.946193 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:35:22.946158    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37b1a86a_13db_4609_923c_fbaf9cd4f8da.slice/crio-75fa55605d0ab581346c0d4aef56056c27e5c72ef444a1c0005864b44fccc543 WatchSource:0}: Error finding container 75fa55605d0ab581346c0d4aef56056c27e5c72ef444a1c0005864b44fccc543: Status 404 returned error can't find the container with id 75fa55605d0ab581346c0d4aef56056c27e5c72ef444a1c0005864b44fccc543
Jul 17 09:35:23.286137 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:23.286104    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cf56599d4-fz8k4" event={"ID":"37b1a86a-13db-4609-923c-fbaf9cd4f8da","Type":"ContainerStarted","Data":"75fa55605d0ab581346c0d4aef56056c27e5c72ef444a1c0005864b44fccc543"}
Jul 17 09:35:26.304421 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:26.304387    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cf56599d4-fz8k4" event={"ID":"37b1a86a-13db-4609-923c-fbaf9cd4f8da","Type":"ContainerStarted","Data":"70f02dda6cc874645b48f7ec2a49c926b6f3a3bcc129268133b90aeea178fed0"}
Jul 17 09:35:26.304775 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:26.304459    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7cf56599d4-fz8k4"
Jul 17 09:35:26.327776 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:26.327730    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7cf56599d4-fz8k4" podStartSLOduration=1.859506061 podStartE2EDuration="4.327718922s" totalImagesPullingTime="2.468212861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:35:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:35:26.325333533 +0000 UTC m=+523.907971910" watchObservedRunningTime="2026-07-17 09:35:26.327718922 +0000 UTC m=+523.910357297"
Jul 17 09:35:37.313337 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:37.313233    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7cf56599d4-fz8k4"
Jul 17 09:35:37.504380 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:37.504349    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7cf56599d4-fz8k4"]
Jul 17 09:35:37.504547 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:37.504523    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7cf56599d4-fz8k4" podUID="37b1a86a-13db-4609-923c-fbaf9cd4f8da" containerName="manager" containerID="cri-o://70f02dda6cc874645b48f7ec2a49c926b6f3a3bcc129268133b90aeea178fed0" gracePeriod=30
Jul 17 09:35:37.754212 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:37.754189    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cf56599d4-fz8k4"
Jul 17 09:35:37.825038 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:37.825009    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/37b1a86a-13db-4609-923c-fbaf9cd4f8da-webhook-cert\") pod \"37b1a86a-13db-4609-923c-fbaf9cd4f8da\" (UID: \"37b1a86a-13db-4609-923c-fbaf9cd4f8da\") "
Jul 17 09:35:37.825231 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:37.825115    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lmp8\" (UniqueName: \"kubernetes.io/projected/37b1a86a-13db-4609-923c-fbaf9cd4f8da-kube-api-access-4lmp8\") pod \"37b1a86a-13db-4609-923c-fbaf9cd4f8da\" (UID: \"37b1a86a-13db-4609-923c-fbaf9cd4f8da\") "
Jul 17 09:35:37.827346 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:37.827315    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37b1a86a-13db-4609-923c-fbaf9cd4f8da-kube-api-access-4lmp8" (OuterVolumeSpecName: "kube-api-access-4lmp8") pod "37b1a86a-13db-4609-923c-fbaf9cd4f8da" (UID: "37b1a86a-13db-4609-923c-fbaf9cd4f8da"). InnerVolumeSpecName "kube-api-access-4lmp8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:35:37.827346 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:37.827334    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37b1a86a-13db-4609-923c-fbaf9cd4f8da-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "37b1a86a-13db-4609-923c-fbaf9cd4f8da" (UID: "37b1a86a-13db-4609-923c-fbaf9cd4f8da"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:35:37.926199 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:37.926166    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lmp8\" (UniqueName: \"kubernetes.io/projected/37b1a86a-13db-4609-923c-fbaf9cd4f8da-kube-api-access-4lmp8\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:35:37.926199 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:37.926198    2574 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/37b1a86a-13db-4609-923c-fbaf9cd4f8da-webhook-cert\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:35:38.364547 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:38.364455    2574 generic.go:356] "Generic (PLEG): container finished" podID="37b1a86a-13db-4609-923c-fbaf9cd4f8da" containerID="70f02dda6cc874645b48f7ec2a49c926b6f3a3bcc129268133b90aeea178fed0" exitCode=0
Jul 17 09:35:38.364547 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:38.364503    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cf56599d4-fz8k4"
Jul 17 09:35:38.364547 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:38.364523    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cf56599d4-fz8k4" event={"ID":"37b1a86a-13db-4609-923c-fbaf9cd4f8da","Type":"ContainerDied","Data":"70f02dda6cc874645b48f7ec2a49c926b6f3a3bcc129268133b90aeea178fed0"}
Jul 17 09:35:38.365070 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:38.364576    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cf56599d4-fz8k4" event={"ID":"37b1a86a-13db-4609-923c-fbaf9cd4f8da","Type":"ContainerDied","Data":"75fa55605d0ab581346c0d4aef56056c27e5c72ef444a1c0005864b44fccc543"}
Jul 17 09:35:38.365070 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:38.364606    2574 scope.go:117] "RemoveContainer" containerID="70f02dda6cc874645b48f7ec2a49c926b6f3a3bcc129268133b90aeea178fed0"
Jul 17 09:35:38.374062 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:38.374048    2574 scope.go:117] "RemoveContainer" containerID="70f02dda6cc874645b48f7ec2a49c926b6f3a3bcc129268133b90aeea178fed0"
Jul 17 09:35:38.374312 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:35:38.374294    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70f02dda6cc874645b48f7ec2a49c926b6f3a3bcc129268133b90aeea178fed0\": container with ID starting with 70f02dda6cc874645b48f7ec2a49c926b6f3a3bcc129268133b90aeea178fed0 not found: ID does not exist" containerID="70f02dda6cc874645b48f7ec2a49c926b6f3a3bcc129268133b90aeea178fed0"
Jul 17 09:35:38.374359 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:38.374320    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70f02dda6cc874645b48f7ec2a49c926b6f3a3bcc129268133b90aeea178fed0"} err="failed to get container status \"70f02dda6cc874645b48f7ec2a49c926b6f3a3bcc129268133b90aeea178fed0\": rpc error: code = NotFound desc = could not find container \"70f02dda6cc874645b48f7ec2a49c926b6f3a3bcc129268133b90aeea178fed0\": container with ID starting with 70f02dda6cc874645b48f7ec2a49c926b6f3a3bcc129268133b90aeea178fed0 not found: ID does not exist"
Jul 17 09:35:38.385715 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:38.385693    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7cf56599d4-fz8k4"]
Jul 17 09:35:38.390208 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:38.390187    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7cf56599d4-fz8k4"]
Jul 17 09:35:38.898298 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:38.898269    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37b1a86a-13db-4609-923c-fbaf9cd4f8da" path="/var/lib/kubelet/pods/37b1a86a-13db-4609-923c-fbaf9cd4f8da/volumes"
Jul 17 09:35:57.650993 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:57.650958    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-9b5xh"]
Jul 17 09:35:57.651410 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:57.651395    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37b1a86a-13db-4609-923c-fbaf9cd4f8da" containerName="manager"
Jul 17 09:35:57.651450 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:57.651412    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="37b1a86a-13db-4609-923c-fbaf9cd4f8da" containerName="manager"
Jul 17 09:35:57.651483 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:57.651478    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="37b1a86a-13db-4609-923c-fbaf9cd4f8da" containerName="manager"
Jul 17 09:35:57.671465 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:57.671433    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-9b5xh"]
Jul 17 09:35:57.671635 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:57.671562    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-9b5xh"
Jul 17 09:35:57.674420 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:57.674395    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 17 09:35:57.674571 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:57.674435    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-l8dgk\""
Jul 17 09:35:57.675203 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:57.675186    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-wpz7b"]
Jul 17 09:35:57.677928 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:57.677903    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-wpz7b"
Jul 17 09:35:57.686829 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:57.686805    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-wpz7b"]
Jul 17 09:35:57.797076 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:57.797042    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptfrq\" (UniqueName: \"kubernetes.io/projected/73bdc8e2-2f1e-41df-8396-cea33c4fd3fd-kube-api-access-ptfrq\") pod \"payload-pre-processing-7677b774bb-wpz7b\" (UID: \"73bdc8e2-2f1e-41df-8396-cea33c4fd3fd\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-wpz7b"
Jul 17 09:35:57.797296 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:57.797084    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40ad3c79-63ba-46bd-ab13-97648ab5d574-plugins-config-volume\") pod \"payload-processing-5bf47cd988-9b5xh\" (UID: \"40ad3c79-63ba-46bd-ab13-97648ab5d574\") " pod="openshift-ingress/payload-processing-5bf47cd988-9b5xh"
Jul 17 09:35:57.797296 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:57.797123    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fswl\" (UniqueName: \"kubernetes.io/projected/40ad3c79-63ba-46bd-ab13-97648ab5d574-kube-api-access-8fswl\") pod \"payload-processing-5bf47cd988-9b5xh\" (UID: \"40ad3c79-63ba-46bd-ab13-97648ab5d574\") " pod="openshift-ingress/payload-processing-5bf47cd988-9b5xh"
Jul 17 09:35:57.797296 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:57.797137    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73bdc8e2-2f1e-41df-8396-cea33c4fd3fd-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-wpz7b\" (UID: \"73bdc8e2-2f1e-41df-8396-cea33c4fd3fd\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-wpz7b"
Jul 17 09:35:57.898013 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:57.897980    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptfrq\" (UniqueName: \"kubernetes.io/projected/73bdc8e2-2f1e-41df-8396-cea33c4fd3fd-kube-api-access-ptfrq\") pod \"payload-pre-processing-7677b774bb-wpz7b\" (UID: \"73bdc8e2-2f1e-41df-8396-cea33c4fd3fd\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-wpz7b"
Jul 17 09:35:57.898013 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:57.898028    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40ad3c79-63ba-46bd-ab13-97648ab5d574-plugins-config-volume\") pod \"payload-processing-5bf47cd988-9b5xh\" (UID: \"40ad3c79-63ba-46bd-ab13-97648ab5d574\") " pod="openshift-ingress/payload-processing-5bf47cd988-9b5xh"
Jul 17 09:35:57.898283 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:57.898077    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fswl\" (UniqueName: \"kubernetes.io/projected/40ad3c79-63ba-46bd-ab13-97648ab5d574-kube-api-access-8fswl\") pod \"payload-processing-5bf47cd988-9b5xh\" (UID: \"40ad3c79-63ba-46bd-ab13-97648ab5d574\") " pod="openshift-ingress/payload-processing-5bf47cd988-9b5xh"
Jul 17 09:35:57.898283 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:57.898099    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73bdc8e2-2f1e-41df-8396-cea33c4fd3fd-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-wpz7b\" (UID: \"73bdc8e2-2f1e-41df-8396-cea33c4fd3fd\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-wpz7b"
Jul 17 09:35:57.898834 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:57.898809    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73bdc8e2-2f1e-41df-8396-cea33c4fd3fd-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-wpz7b\" (UID: \"73bdc8e2-2f1e-41df-8396-cea33c4fd3fd\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-wpz7b"
Jul 17 09:35:57.898924 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:57.898871    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40ad3c79-63ba-46bd-ab13-97648ab5d574-plugins-config-volume\") pod \"payload-processing-5bf47cd988-9b5xh\" (UID: \"40ad3c79-63ba-46bd-ab13-97648ab5d574\") " pod="openshift-ingress/payload-processing-5bf47cd988-9b5xh"
Jul 17 09:35:57.907178 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:57.907122    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptfrq\" (UniqueName: \"kubernetes.io/projected/73bdc8e2-2f1e-41df-8396-cea33c4fd3fd-kube-api-access-ptfrq\") pod \"payload-pre-processing-7677b774bb-wpz7b\" (UID: \"73bdc8e2-2f1e-41df-8396-cea33c4fd3fd\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-wpz7b"
Jul 17 09:35:57.907545 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:57.907527    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fswl\" (UniqueName: \"kubernetes.io/projected/40ad3c79-63ba-46bd-ab13-97648ab5d574-kube-api-access-8fswl\") pod \"payload-processing-5bf47cd988-9b5xh\" (UID: \"40ad3c79-63ba-46bd-ab13-97648ab5d574\") " pod="openshift-ingress/payload-processing-5bf47cd988-9b5xh"
Jul 17 09:35:57.983462 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:57.983440    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-9b5xh"
Jul 17 09:35:57.991994 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:57.991979    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-wpz7b"
Jul 17 09:35:58.155508 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:58.154020    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-9b5xh"]
Jul 17 09:35:58.178948 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:58.178899    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-wpz7b"]
Jul 17 09:35:58.184729 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:35:58.184695    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73bdc8e2_2f1e_41df_8396_cea33c4fd3fd.slice/crio-1be85872bd5e4327bad035a0558cc622a470c6f6ca228d2ea258a879c10f42de WatchSource:0}: Error finding container 1be85872bd5e4327bad035a0558cc622a470c6f6ca228d2ea258a879c10f42de: Status 404 returned error can't find the container with id 1be85872bd5e4327bad035a0558cc622a470c6f6ca228d2ea258a879c10f42de
Jul 17 09:35:58.463449 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:58.463352    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-9b5xh" event={"ID":"40ad3c79-63ba-46bd-ab13-97648ab5d574","Type":"ContainerStarted","Data":"111c3a335bd4195cc8b984ff06b6269ab2a861c71b59a41ce2691b21fa1937b5"}
Jul 17 09:35:58.464492 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:35:58.464469    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-wpz7b" event={"ID":"73bdc8e2-2f1e-41df-8396-cea33c4fd3fd","Type":"ContainerStarted","Data":"1be85872bd5e4327bad035a0558cc622a470c6f6ca228d2ea258a879c10f42de"}
Jul 17 09:36:01.486315 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:01.486276    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-wpz7b" event={"ID":"73bdc8e2-2f1e-41df-8396-cea33c4fd3fd","Type":"ContainerStarted","Data":"2cd62b173c5a829f3366ed56f251fc8317a227509c4305737fed0346abd01cc1"}
Jul 17 09:36:01.486738 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:01.486344    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-wpz7b"
Jul 17 09:36:01.487756 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:01.487735    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-9b5xh" event={"ID":"40ad3c79-63ba-46bd-ab13-97648ab5d574","Type":"ContainerStarted","Data":"a503fad6c379205c1e2519c4680dbefb03fda7a5eb0f7eeda08b87d00d2fb3f1"}
Jul 17 09:36:01.487880 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:01.487864    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-9b5xh"
Jul 17 09:36:01.501612 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:01.501522    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-wpz7b" podStartSLOduration=2.229420365 podStartE2EDuration="4.501506199s" totalImagesPullingTime="2.272085834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:35:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:36:01.500596001 +0000 UTC m=+559.083234378" watchObservedRunningTime="2026-07-17 09:36:01.501506199 +0000 UTC m=+559.084144576"
Jul 17 09:36:01.520371 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:01.520327    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-9b5xh" podStartSLOduration=2.228724156 podStartE2EDuration="4.520308774s" totalImagesPullingTime="2.291584618s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:35:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:36:01.516818483 +0000 UTC m=+559.099456860" watchObservedRunningTime="2026-07-17 09:36:01.520308774 +0000 UTC m=+559.102947152"
Jul 17 09:36:05.510602 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:05.510572    2574 generic.go:356] "Generic (PLEG): container finished" podID="aff066d2-5135-49e2-b041-39424acf8fb6" containerID="dc17cca57d50bcf584724bd24d7d8ea296bef0a3a26b9326972563dedab1b177" exitCode=0
Jul 17 09:36:05.511078 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:05.510656    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn" event={"ID":"aff066d2-5135-49e2-b041-39424acf8fb6","Type":"ContainerDied","Data":"dc17cca57d50bcf584724bd24d7d8ea296bef0a3a26b9326972563dedab1b177"}
Jul 17 09:36:05.511141 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:05.511130    2574 scope.go:117] "RemoveContainer" containerID="dc17cca57d50bcf584724bd24d7d8ea296bef0a3a26b9326972563dedab1b177"
Jul 17 09:36:05.512107 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:05.512081    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 09:36:05.512188 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:05.512145    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 09:36:05.512188 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:05.512179    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 09:36:06.520717 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:06.520669    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn" event={"ID":"aff066d2-5135-49e2-b041-39424acf8fb6","Type":"ContainerStarted","Data":"63bbe849411139966136d2dd8fc11bd06b5e620bed709ff859f4799585c7159b"}
Jul 17 09:36:06.614474 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:06.614439    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:36:06.615048 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:06.615019    2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-x5rwn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Jul 17 09:36:06.615176 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:06.615076    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn" podUID="aff066d2-5135-49e2-b041-39424acf8fb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Jul 17 09:36:07.601815 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:07.601776    2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-x5rwn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Jul 17 09:36:07.602350 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:07.601837    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn" podUID="aff066d2-5135-49e2-b041-39424acf8fb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Jul 17 09:36:08.601310 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:08.601224    2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-x5rwn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Jul 17 09:36:08.601582 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:08.601322    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn" podUID="aff066d2-5135-49e2-b041-39424acf8fb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Jul 17 09:36:09.601937 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:09.601905    2574 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-x5rwn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body=
Jul 17 09:36:09.602475 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:09.602445    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn" podUID="aff066d2-5135-49e2-b041-39424acf8fb6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused"
Jul 17 09:36:10.605467 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:10.605439    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:36:10.605932 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:10.605774    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:36:10.606628 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:10.606610    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-x5rwn"
Jul 17 09:36:12.496153 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:12.496126    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-wpz7b"
Jul 17 09:36:12.496525 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:12.496187    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-9b5xh"
Jul 17 09:36:17.271369 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:17.271338    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5"]
Jul 17 09:36:17.274894 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:17.274869    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5"
Jul 17 09:36:17.278341 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:17.278312    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 17 09:36:17.279690 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:17.279675    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-thd5v\""
Jul 17 09:36:17.279817 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:17.279752    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 17 09:36:17.279993 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:17.279961    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 17 09:36:17.286601 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:17.286576    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5"]
Jul 17 09:36:17.376186 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:17.376157    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5\" (UID: \"4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5"
Jul 17 09:36:17.376365 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:17.376233    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5\" (UID: \"4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5"
Jul 17 09:36:17.376365 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:17.376300    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5\" (UID: \"4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5"
Jul 17 09:36:17.376365 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:17.376340    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5\" (UID: \"4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5"
Jul 17 09:36:17.376531 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:17.376372    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pdpz\" (UniqueName: \"kubernetes.io/projected/4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2-kube-api-access-4pdpz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5\" (UID: \"4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5"
Jul 17 09:36:17.376531 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:17.376417    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5\" (UID: \"4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5"
Jul 17 09:36:17.477019 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:17.476987    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5\" (UID: \"4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5"
Jul 17 09:36:17.477208 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:17.477028    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pdpz\" (UniqueName: \"kubernetes.io/projected/4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2-kube-api-access-4pdpz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5\" (UID: \"4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5"
Jul 17 09:36:17.477208 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:17.477060    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5\" (UID: \"4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5"
Jul 17 09:36:17.477208 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:17.477094    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5\" (UID: \"4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5"
Jul 17 09:36:17.477208 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:17.477144    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5\" (UID: \"4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5"
Jul 17 09:36:17.477208 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:17.477174    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5\" (UID: \"4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5"
Jul 17 09:36:17.477509 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:17.477471    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5\" (UID: \"4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5"
Jul 17 09:36:17.477509 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:17.477502    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5\" (UID: \"4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5"
Jul 17 09:36:17.477615 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:17.477551    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5\" (UID: \"4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5"
Jul 17 09:36:17.479588 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:17.479559    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5\" (UID: \"4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5"
Jul 17 09:36:17.479787 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:17.479768    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5\" (UID: \"4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5"
Jul 17 09:36:17.493726 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:17.493706    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pdpz\" (UniqueName: \"kubernetes.io/projected/4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2-kube-api-access-4pdpz\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5\" (UID: \"4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5"
Jul 17 09:36:17.586284 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:17.586200    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5"
Jul 17 09:36:17.723009 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:17.722983    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5"]
Jul 17 09:36:17.724552 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:36:17.724531    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c8c40db_54fe_41b4_a87c_5ce9f5c4d5c2.slice/crio-adb4dcb19078f9d7fa93527fe5b49d23790a1ab6bae8567934bb01dbceb12e80 WatchSource:0}: Error finding container adb4dcb19078f9d7fa93527fe5b49d23790a1ab6bae8567934bb01dbceb12e80: Status 404 returned error can't find the container with id adb4dcb19078f9d7fa93527fe5b49d23790a1ab6bae8567934bb01dbceb12e80
Jul 17 09:36:18.585515 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:18.585482    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5" event={"ID":"4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2","Type":"ContainerStarted","Data":"adb4dcb19078f9d7fa93527fe5b49d23790a1ab6bae8567934bb01dbceb12e80"}
Jul 17 09:36:19.593117 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:19.593073    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5" event={"ID":"4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2","Type":"ContainerStarted","Data":"30d2f6cd6a24597ae8ad88bd1cc16a194de2d6f13466b57dc45828719083cc8f"}
Jul 17 09:36:19.612865 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:19.612815    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5" podStartSLOduration=1.314165315 podStartE2EDuration="2.612797375s" totalImagesPullingTime="1.29863206s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:36:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:36:19.611148613 +0000 UTC m=+577.193786989" watchObservedRunningTime="2026-07-17 09:36:19.612797375 +0000 UTC m=+577.195435751"
Jul 17 09:36:27.586882 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:27.586844    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5"
Jul 17 09:36:27.587234 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:27.586919    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5"
Jul 17 09:36:27.599262 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:27.599226    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5"
Jul 17 09:36:27.646377 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:27.646354    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5"
Jul 17 09:36:42.845459 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:42.845421    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dh564_84db06ab-87e2-43b3-82d0-13f06ba86d06/ovn-acl-logging/0.log"
Jul 17 09:36:42.847002 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:42.846980    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dh564_84db06ab-87e2-43b3-82d0-13f06ba86d06/ovn-acl-logging/0.log"
Jul 17 09:36:50.957769 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:50.957738    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk"]
Jul 17 09:36:50.964622 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:50.964602    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk"
Jul 17 09:36:50.967506 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:50.967482    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 17 09:36:50.971931 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:50.971909    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk"]
Jul 17 09:36:51.034597 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:51.034569    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa111f9b-d374-4991-a041-8c05c7c45011-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nt9tk\" (UID: \"fa111f9b-d374-4991-a041-8c05c7c45011\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk"
Jul 17 09:36:51.034597 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:51.034600    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa111f9b-d374-4991-a041-8c05c7c45011-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nt9tk\" (UID: \"fa111f9b-d374-4991-a041-8c05c7c45011\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk"
Jul 17 09:36:51.034816 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:51.034624    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa111f9b-d374-4991-a041-8c05c7c45011-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nt9tk\" (UID: \"fa111f9b-d374-4991-a041-8c05c7c45011\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk"
Jul 17 09:36:51.034816 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:51.034702    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa111f9b-d374-4991-a041-8c05c7c45011-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nt9tk\" (UID: \"fa111f9b-d374-4991-a041-8c05c7c45011\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk"
Jul 17 09:36:51.034816 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:51.034783    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79qbg\" (UniqueName: \"kubernetes.io/projected/fa111f9b-d374-4991-a041-8c05c7c45011-kube-api-access-79qbg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nt9tk\" (UID: \"fa111f9b-d374-4991-a041-8c05c7c45011\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk"
Jul 17 09:36:51.034946 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:51.034823    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa111f9b-d374-4991-a041-8c05c7c45011-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nt9tk\" (UID: \"fa111f9b-d374-4991-a041-8c05c7c45011\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk"
Jul 17 09:36:51.135393 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:51.135367    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa111f9b-d374-4991-a041-8c05c7c45011-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nt9tk\" (UID: \"fa111f9b-d374-4991-a041-8c05c7c45011\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk"
Jul 17 09:36:51.135530 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:51.135414    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa111f9b-d374-4991-a041-8c05c7c45011-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nt9tk\" (UID: \"fa111f9b-d374-4991-a041-8c05c7c45011\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk"
Jul 17 09:36:51.135530 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:51.135450    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79qbg\" (UniqueName: \"kubernetes.io/projected/fa111f9b-d374-4991-a041-8c05c7c45011-kube-api-access-79qbg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nt9tk\" (UID: \"fa111f9b-d374-4991-a041-8c05c7c45011\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk"
Jul 17 09:36:51.135530 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:51.135479    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa111f9b-d374-4991-a041-8c05c7c45011-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nt9tk\" (UID: \"fa111f9b-d374-4991-a041-8c05c7c45011\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk"
Jul 17 09:36:51.135530 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:51.135509    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa111f9b-d374-4991-a041-8c05c7c45011-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nt9tk\" (UID: \"fa111f9b-d374-4991-a041-8c05c7c45011\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk"
Jul 17 09:36:51.135530 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:51.135523    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa111f9b-d374-4991-a041-8c05c7c45011-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nt9tk\" (UID: \"fa111f9b-d374-4991-a041-8c05c7c45011\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk"
Jul 17 09:36:51.135953 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:51.135920    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa111f9b-d374-4991-a041-8c05c7c45011-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nt9tk\" (UID: \"fa111f9b-d374-4991-a041-8c05c7c45011\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk"
Jul 17 09:36:51.135953 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:51.135938    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa111f9b-d374-4991-a041-8c05c7c45011-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nt9tk\" (UID: \"fa111f9b-d374-4991-a041-8c05c7c45011\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk"
Jul 17 09:36:51.136095 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:51.136008    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa111f9b-d374-4991-a041-8c05c7c45011-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nt9tk\" (UID: \"fa111f9b-d374-4991-a041-8c05c7c45011\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk"
Jul 17 09:36:51.137865 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:51.137842    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa111f9b-d374-4991-a041-8c05c7c45011-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nt9tk\" (UID: \"fa111f9b-d374-4991-a041-8c05c7c45011\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk"
Jul 17 09:36:51.138129 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:51.138112    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa111f9b-d374-4991-a041-8c05c7c45011-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nt9tk\" (UID: \"fa111f9b-d374-4991-a041-8c05c7c45011\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk"
Jul 17 09:36:51.143272 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:51.143226    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79qbg\" (UniqueName: \"kubernetes.io/projected/fa111f9b-d374-4991-a041-8c05c7c45011-kube-api-access-79qbg\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nt9tk\" (UID: \"fa111f9b-d374-4991-a041-8c05c7c45011\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk"
Jul 17 09:36:51.276732 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:51.276676    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk"
Jul 17 09:36:51.412138 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:51.412117    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk"]
Jul 17 09:36:51.414379 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:36:51.414347    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa111f9b_d374_4991_a041_8c05c7c45011.slice/crio-fd588427e456f789ef41b48384137d7282ef5cf4741a56026d44449dd39e603d WatchSource:0}: Error finding container fd588427e456f789ef41b48384137d7282ef5cf4741a56026d44449dd39e603d: Status 404 returned error can't find the container with id fd588427e456f789ef41b48384137d7282ef5cf4741a56026d44449dd39e603d
Jul 17 09:36:51.756477 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:51.756434    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk" event={"ID":"fa111f9b-d374-4991-a041-8c05c7c45011","Type":"ContainerStarted","Data":"4bb605032a33ac606d16720c6ae03ed5bb538904b1ed9f01bc062c1424f911eb"}
Jul 17 09:36:51.756606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:51.756502    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk" event={"ID":"fa111f9b-d374-4991-a041-8c05c7c45011","Type":"ContainerStarted","Data":"fd588427e456f789ef41b48384137d7282ef5cf4741a56026d44449dd39e603d"}
Jul 17 09:36:51.775009 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:36:51.774958    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk" podStartSLOduration=1.5447941250000001 podStartE2EDuration="1.77493766s" totalImagesPullingTime="230.143535ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:36:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:36:51.773039136 +0000 UTC m=+609.355677512" watchObservedRunningTime="2026-07-17 09:36:51.77493766 +0000 UTC m=+609.357576037"
Jul 17 09:37:01.276871 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:37:01.276838    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk"
Jul 17 09:37:01.277330 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:37:01.276905    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk"
Jul 17 09:37:01.289085 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:37:01.289058    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk"
Jul 17 09:37:01.818333 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:37:01.818304    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nt9tk"
Jul 17 09:40:12.455485 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:12.455387    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c5c6b98f-qr6kt"]
Jul 17 09:40:12.459256 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:12.459215    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c5c6b98f-qr6kt"
Jul 17 09:40:12.463372 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:12.463341    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 17 09:40:12.463499 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:12.463375    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 17 09:40:12.463499 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:12.463348    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jc8mw\""
Jul 17 09:40:12.469346 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:12.469316    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c5c6b98f-qr6kt"]
Jul 17 09:40:12.556349 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:12.556317    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szt79\" (UniqueName: \"kubernetes.io/projected/6892b297-9157-4346-9041-d7a0e9f830e4-kube-api-access-szt79\") pod \"maas-controller-6c5c6b98f-qr6kt\" (UID: \"6892b297-9157-4346-9041-d7a0e9f830e4\") " pod="opendatahub/maas-controller-6c5c6b98f-qr6kt"
Jul 17 09:40:12.556499 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:12.556361    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6892b297-9157-4346-9041-d7a0e9f830e4-webhook-cert\") pod \"maas-controller-6c5c6b98f-qr6kt\" (UID: \"6892b297-9157-4346-9041-d7a0e9f830e4\") " pod="opendatahub/maas-controller-6c5c6b98f-qr6kt"
Jul 17 09:40:12.657672 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:12.657637    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szt79\" (UniqueName: \"kubernetes.io/projected/6892b297-9157-4346-9041-d7a0e9f830e4-kube-api-access-szt79\") pod \"maas-controller-6c5c6b98f-qr6kt\" (UID: \"6892b297-9157-4346-9041-d7a0e9f830e4\") " pod="opendatahub/maas-controller-6c5c6b98f-qr6kt"
Jul 17 09:40:12.657857 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:12.657685    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6892b297-9157-4346-9041-d7a0e9f830e4-webhook-cert\") pod \"maas-controller-6c5c6b98f-qr6kt\" (UID: \"6892b297-9157-4346-9041-d7a0e9f830e4\") " pod="opendatahub/maas-controller-6c5c6b98f-qr6kt"
Jul 17 09:40:12.660434 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:12.660398    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6892b297-9157-4346-9041-d7a0e9f830e4-webhook-cert\") pod \"maas-controller-6c5c6b98f-qr6kt\" (UID: \"6892b297-9157-4346-9041-d7a0e9f830e4\") " pod="opendatahub/maas-controller-6c5c6b98f-qr6kt"
Jul 17 09:40:12.666783 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:12.666759    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-szt79\" (UniqueName: \"kubernetes.io/projected/6892b297-9157-4346-9041-d7a0e9f830e4-kube-api-access-szt79\") pod \"maas-controller-6c5c6b98f-qr6kt\" (UID: \"6892b297-9157-4346-9041-d7a0e9f830e4\") " pod="opendatahub/maas-controller-6c5c6b98f-qr6kt"
Jul 17 09:40:12.770943 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:12.770851    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c5c6b98f-qr6kt"
Jul 17 09:40:12.909155 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:12.909131    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c5c6b98f-qr6kt"]
Jul 17 09:40:12.911755 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:40:12.911727    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6892b297_9157_4346_9041_d7a0e9f830e4.slice/crio-c09bb948e755de0bcf45ac89f42d1586055cd77210ffda2515d0ee90346c6069 WatchSource:0}: Error finding container c09bb948e755de0bcf45ac89f42d1586055cd77210ffda2515d0ee90346c6069: Status 404 returned error can't find the container with id c09bb948e755de0bcf45ac89f42d1586055cd77210ffda2515d0ee90346c6069
Jul 17 09:40:12.913198 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:12.913174    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 09:40:13.790921 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:13.790876    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c5c6b98f-qr6kt" event={"ID":"6892b297-9157-4346-9041-d7a0e9f830e4","Type":"ContainerStarted","Data":"03737a19e4f2863b46713bb9925bf0fc438ffc99e1f4115622ba2b0479efc874"}
Jul 17 09:40:13.791354 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:13.790957    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c5c6b98f-qr6kt" event={"ID":"6892b297-9157-4346-9041-d7a0e9f830e4","Type":"ContainerStarted","Data":"c09bb948e755de0bcf45ac89f42d1586055cd77210ffda2515d0ee90346c6069"}
Jul 17 09:40:13.791354 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:13.791041    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c5c6b98f-qr6kt"
Jul 17 09:40:13.809954 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:13.809907    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c5c6b98f-qr6kt" podStartSLOduration=1.411806027 podStartE2EDuration="1.809895097s" totalImagesPullingTime="398.08907ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:40:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:40:13.808766928 +0000 UTC m=+811.391405323" watchObservedRunningTime="2026-07-17 09:40:13.809895097 +0000 UTC m=+811.392533472"
Jul 17 09:40:24.798220 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:24.798188    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c5c6b98f-qr6kt"
Jul 17 09:40:29.446468 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:29.446425    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c5c6b98f-qr6kt"]
Jul 17 09:40:29.446982 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:29.446753    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6c5c6b98f-qr6kt" podUID="6892b297-9157-4346-9041-d7a0e9f830e4" containerName="manager" containerID="cri-o://03737a19e4f2863b46713bb9925bf0fc438ffc99e1f4115622ba2b0479efc874" gracePeriod=30
Jul 17 09:40:29.709603 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:29.709525    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c5c6b98f-qr6kt"
Jul 17 09:40:29.822937 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:29.822894    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-szt79\" (UniqueName: \"kubernetes.io/projected/6892b297-9157-4346-9041-d7a0e9f830e4-kube-api-access-szt79\") pod \"6892b297-9157-4346-9041-d7a0e9f830e4\" (UID: \"6892b297-9157-4346-9041-d7a0e9f830e4\") "
Jul 17 09:40:29.823125 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:29.823029    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6892b297-9157-4346-9041-d7a0e9f830e4-webhook-cert\") pod \"6892b297-9157-4346-9041-d7a0e9f830e4\" (UID: \"6892b297-9157-4346-9041-d7a0e9f830e4\") "
Jul 17 09:40:29.825004 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:29.824971    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6892b297-9157-4346-9041-d7a0e9f830e4-kube-api-access-szt79" (OuterVolumeSpecName: "kube-api-access-szt79") pod "6892b297-9157-4346-9041-d7a0e9f830e4" (UID: "6892b297-9157-4346-9041-d7a0e9f830e4"). InnerVolumeSpecName "kube-api-access-szt79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:40:29.825119 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:29.825043    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6892b297-9157-4346-9041-d7a0e9f830e4-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6892b297-9157-4346-9041-d7a0e9f830e4" (UID: "6892b297-9157-4346-9041-d7a0e9f830e4"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 17 09:40:29.882590 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:29.882556    2574 generic.go:356] "Generic (PLEG): container finished" podID="6892b297-9157-4346-9041-d7a0e9f830e4" containerID="03737a19e4f2863b46713bb9925bf0fc438ffc99e1f4115622ba2b0479efc874" exitCode=0
Jul 17 09:40:29.882752 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:29.882599    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c5c6b98f-qr6kt"
Jul 17 09:40:29.882752 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:29.882646    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c5c6b98f-qr6kt" event={"ID":"6892b297-9157-4346-9041-d7a0e9f830e4","Type":"ContainerDied","Data":"03737a19e4f2863b46713bb9925bf0fc438ffc99e1f4115622ba2b0479efc874"}
Jul 17 09:40:29.882752 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:29.882699    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c5c6b98f-qr6kt" event={"ID":"6892b297-9157-4346-9041-d7a0e9f830e4","Type":"ContainerDied","Data":"c09bb948e755de0bcf45ac89f42d1586055cd77210ffda2515d0ee90346c6069"}
Jul 17 09:40:29.882752 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:29.882721    2574 scope.go:117] "RemoveContainer" containerID="03737a19e4f2863b46713bb9925bf0fc438ffc99e1f4115622ba2b0479efc874"
Jul 17 09:40:29.891950 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:29.891933    2574 scope.go:117] "RemoveContainer" containerID="03737a19e4f2863b46713bb9925bf0fc438ffc99e1f4115622ba2b0479efc874"
Jul 17 09:40:29.892198 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:40:29.892181    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03737a19e4f2863b46713bb9925bf0fc438ffc99e1f4115622ba2b0479efc874\": container with ID starting with 03737a19e4f2863b46713bb9925bf0fc438ffc99e1f4115622ba2b0479efc874 not found: ID does not exist" containerID="03737a19e4f2863b46713bb9925bf0fc438ffc99e1f4115622ba2b0479efc874"
Jul 17 09:40:29.892293 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:29.892207    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03737a19e4f2863b46713bb9925bf0fc438ffc99e1f4115622ba2b0479efc874"} err="failed to get container status \"03737a19e4f2863b46713bb9925bf0fc438ffc99e1f4115622ba2b0479efc874\": rpc error: code = NotFound desc = could not find container \"03737a19e4f2863b46713bb9925bf0fc438ffc99e1f4115622ba2b0479efc874\": container with ID starting with 03737a19e4f2863b46713bb9925bf0fc438ffc99e1f4115622ba2b0479efc874 not found: ID does not exist"
Jul 17 09:40:29.905290 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:29.905265    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c5c6b98f-qr6kt"]
Jul 17 09:40:29.910755 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:29.910731    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6c5c6b98f-qr6kt"]
Jul 17 09:40:29.923975 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:29.923956    2574 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6892b297-9157-4346-9041-d7a0e9f830e4-webhook-cert\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:40:29.923975 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:29.923976    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-szt79\" (UniqueName: \"kubernetes.io/projected/6892b297-9157-4346-9041-d7a0e9f830e4-kube-api-access-szt79\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:40:30.672581 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:30.672546    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c5c6b98f-rlrg6"]
Jul 17 09:40:30.673044 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:30.673030    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6892b297-9157-4346-9041-d7a0e9f830e4" containerName="manager"
Jul 17 09:40:30.673087 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:30.673046    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="6892b297-9157-4346-9041-d7a0e9f830e4" containerName="manager"
Jul 17 09:40:30.673136 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:30.673127    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="6892b297-9157-4346-9041-d7a0e9f830e4" containerName="manager"
Jul 17 09:40:30.677683 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:30.677668    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c5c6b98f-rlrg6"
Jul 17 09:40:30.681904 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:30.681880    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 17 09:40:30.682045 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:30.681948    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jc8mw\""
Jul 17 09:40:30.682045 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:30.681880    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 17 09:40:30.684871 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:30.684849    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c5c6b98f-rlrg6"]
Jul 17 09:40:30.831742 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:30.831713    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7xvv\" (UniqueName: \"kubernetes.io/projected/8ab844e0-7c94-4003-9e42-ada02b05d4e9-kube-api-access-h7xvv\") pod \"maas-controller-6c5c6b98f-rlrg6\" (UID: \"8ab844e0-7c94-4003-9e42-ada02b05d4e9\") " pod="opendatahub/maas-controller-6c5c6b98f-rlrg6"
Jul 17 09:40:30.831915 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:30.831765    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8ab844e0-7c94-4003-9e42-ada02b05d4e9-webhook-cert\") pod \"maas-controller-6c5c6b98f-rlrg6\" (UID: \"8ab844e0-7c94-4003-9e42-ada02b05d4e9\") " pod="opendatahub/maas-controller-6c5c6b98f-rlrg6"
Jul 17 09:40:30.896947 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:30.896913    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6892b297-9157-4346-9041-d7a0e9f830e4" path="/var/lib/kubelet/pods/6892b297-9157-4346-9041-d7a0e9f830e4/volumes"
Jul 17 09:40:30.932496 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:30.932418    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7xvv\" (UniqueName: \"kubernetes.io/projected/8ab844e0-7c94-4003-9e42-ada02b05d4e9-kube-api-access-h7xvv\") pod \"maas-controller-6c5c6b98f-rlrg6\" (UID: \"8ab844e0-7c94-4003-9e42-ada02b05d4e9\") " pod="opendatahub/maas-controller-6c5c6b98f-rlrg6"
Jul 17 09:40:30.932496 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:30.932468    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8ab844e0-7c94-4003-9e42-ada02b05d4e9-webhook-cert\") pod \"maas-controller-6c5c6b98f-rlrg6\" (UID: \"8ab844e0-7c94-4003-9e42-ada02b05d4e9\") " pod="opendatahub/maas-controller-6c5c6b98f-rlrg6"
Jul 17 09:40:30.934940 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:30.934912    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8ab844e0-7c94-4003-9e42-ada02b05d4e9-webhook-cert\") pod \"maas-controller-6c5c6b98f-rlrg6\" (UID: \"8ab844e0-7c94-4003-9e42-ada02b05d4e9\") " pod="opendatahub/maas-controller-6c5c6b98f-rlrg6"
Jul 17 09:40:30.941004 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:30.940987    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7xvv\" (UniqueName: \"kubernetes.io/projected/8ab844e0-7c94-4003-9e42-ada02b05d4e9-kube-api-access-h7xvv\") pod \"maas-controller-6c5c6b98f-rlrg6\" (UID: \"8ab844e0-7c94-4003-9e42-ada02b05d4e9\") " pod="opendatahub/maas-controller-6c5c6b98f-rlrg6"
Jul 17 09:40:30.990046 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:30.990015    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c5c6b98f-rlrg6"
Jul 17 09:40:31.149347 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:31.149314    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c5c6b98f-rlrg6"]
Jul 17 09:40:31.151390 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:40:31.151360    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ab844e0_7c94_4003_9e42_ada02b05d4e9.slice/crio-3cc24ed2770a59166f254441fd67ba7949af6a0273fcd3e00d6a68d97575849e WatchSource:0}: Error finding container 3cc24ed2770a59166f254441fd67ba7949af6a0273fcd3e00d6a68d97575849e: Status 404 returned error can't find the container with id 3cc24ed2770a59166f254441fd67ba7949af6a0273fcd3e00d6a68d97575849e
Jul 17 09:40:31.895925 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:31.895895    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c5c6b98f-rlrg6" event={"ID":"8ab844e0-7c94-4003-9e42-ada02b05d4e9","Type":"ContainerStarted","Data":"7b6d833d01ef8a85b93910c699a24fa8f25c87d3de0969fc9b8e76f2d278ef38"}
Jul 17 09:40:31.896331 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:31.895949    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6c5c6b98f-rlrg6"
Jul 17 09:40:31.896331 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:31.895972    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6c5c6b98f-rlrg6" event={"ID":"8ab844e0-7c94-4003-9e42-ada02b05d4e9","Type":"ContainerStarted","Data":"3cc24ed2770a59166f254441fd67ba7949af6a0273fcd3e00d6a68d97575849e"}
Jul 17 09:40:31.913423 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:31.913379    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6c5c6b98f-rlrg6" podStartSLOduration=1.390955528 podStartE2EDuration="1.913364352s" totalImagesPullingTime="522.408824ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:40:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:40:31.911722372 +0000 UTC m=+829.494360747" watchObservedRunningTime="2026-07-17 09:40:31.913364352 +0000 UTC m=+829.496002727"
Jul 17 09:40:42.903130 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:40:42.903101    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6c5c6b98f-rlrg6"
Jul 17 09:41:42.881805 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:41:42.881730    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dh564_84db06ab-87e2-43b3-82d0-13f06ba86d06/ovn-acl-logging/0.log"
Jul 17 09:41:42.881805 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:41:42.881751    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dh564_84db06ab-87e2-43b3-82d0-13f06ba86d06/ovn-acl-logging/0.log"
Jul 17 09:45:00.136333 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:00.136292    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738025-npjpr"]
Jul 17 09:45:00.139934 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:00.139917    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738025-npjpr"
Jul 17 09:45:00.142764 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:00.142745    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 09:45:00.150467 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:00.150442    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738025-npjpr"
Jul 17 09:45:00.151326 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:00.151304    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738025-npjpr"]
Jul 17 09:45:00.502740 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:00.502709    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738025-npjpr"]
Jul 17 09:45:00.504513 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:45:00.504488    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1e47955_752c_4b2e_b76f_9d1efaa72250.slice/crio-da35e587a879b8c7ea50690fcc1e07a36371c200a474426f71982a027edc595b WatchSource:0}: Error finding container da35e587a879b8c7ea50690fcc1e07a36371c200a474426f71982a027edc595b: Status 404 returned error can't find the container with id da35e587a879b8c7ea50690fcc1e07a36371c200a474426f71982a027edc595b
Jul 17 09:45:01.340458 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:01.340425    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738025-npjpr" event={"ID":"e1e47955-752c-4b2e-b76f-9d1efaa72250","Type":"ContainerStarted","Data":"da35e587a879b8c7ea50690fcc1e07a36371c200a474426f71982a027edc595b"}
Jul 17 09:45:03.354598 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:03.354560    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738025-npjpr" event={"ID":"e1e47955-752c-4b2e-b76f-9d1efaa72250","Type":"ContainerStarted","Data":"01b0f7877e0123eff14170f661cd4884ee6a626ab555688f884b4fda8f994097"}
Jul 17 09:45:03.369419 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:03.369365    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738025-npjpr" podStartSLOduration=0.965501517 podStartE2EDuration="3.369350258s" totalImagesPullingTime="2.403848741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:45:03.368638383 +0000 UTC m=+1100.951276760" watchObservedRunningTime="2026-07-17 09:45:03.369350258 +0000 UTC m=+1100.951988635"
Jul 17 09:45:04.361946 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:04.361852    2574 generic.go:356] "Generic (PLEG): container finished" podID="e1e47955-752c-4b2e-b76f-9d1efaa72250" containerID="01b0f7877e0123eff14170f661cd4884ee6a626ab555688f884b4fda8f994097" exitCode=0
Jul 17 09:45:04.361946 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:04.361894    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738025-npjpr" event={"ID":"e1e47955-752c-4b2e-b76f-9d1efaa72250","Type":"ContainerDied","Data":"01b0f7877e0123eff14170f661cd4884ee6a626ab555688f884b4fda8f994097"}
Jul 17 09:45:05.502230 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:05.502198    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738025-npjpr"
Jul 17 09:45:06.373617 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:06.373583    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738025-npjpr"
Jul 17 09:45:06.373617 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:06.373603    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738025-npjpr" event={"ID":"e1e47955-752c-4b2e-b76f-9d1efaa72250","Type":"ContainerDied","Data":"da35e587a879b8c7ea50690fcc1e07a36371c200a474426f71982a027edc595b"}
Jul 17 09:45:06.373928 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:06.373658    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="da35e587a879b8c7ea50690fcc1e07a36371c200a474426f71982a027edc595b"
Jul 17 09:45:17.685104 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:17.685065    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lmklg"]
Jul 17 09:45:17.685568 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:17.685294    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lmklg" podUID="f9e42161-af91-4958-bebe-2a87ff299a1a" containerName="manager" containerID="cri-o://4e5b6d9a8bb38d6f5fe068e7a0116fd4f715cc4a9cec30fdf3204db1ecfd3897" gracePeriod=10
Jul 17 09:45:20.906107 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:20.906073    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lmklg" podUID="f9e42161-af91-4958-bebe-2a87ff299a1a" containerName="manager" probeResult="failure" output="Get \"http://10.132.0.41:8081/readyz\": dial tcp 10.132.0.41:8081: connect: connection refused"
Jul 17 09:45:21.047038 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:21.047015    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lmklg"
Jul 17 09:45:21.153919 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:21.153835    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpdwf\" (UniqueName: \"kubernetes.io/projected/f9e42161-af91-4958-bebe-2a87ff299a1a-kube-api-access-qpdwf\") pod \"f9e42161-af91-4958-bebe-2a87ff299a1a\" (UID: \"f9e42161-af91-4958-bebe-2a87ff299a1a\") "
Jul 17 09:45:21.154055 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:21.153935    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9e42161-af91-4958-bebe-2a87ff299a1a-extensions-socket-volume\") pod \"f9e42161-af91-4958-bebe-2a87ff299a1a\" (UID: \"f9e42161-af91-4958-bebe-2a87ff299a1a\") "
Jul 17 09:45:21.154346 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:21.154322    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9e42161-af91-4958-bebe-2a87ff299a1a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f9e42161-af91-4958-bebe-2a87ff299a1a" (UID: "f9e42161-af91-4958-bebe-2a87ff299a1a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 09:45:21.156106 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:21.156087    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9e42161-af91-4958-bebe-2a87ff299a1a-kube-api-access-qpdwf" (OuterVolumeSpecName: "kube-api-access-qpdwf") pod "f9e42161-af91-4958-bebe-2a87ff299a1a" (UID: "f9e42161-af91-4958-bebe-2a87ff299a1a"). InnerVolumeSpecName "kube-api-access-qpdwf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 09:45:21.255647 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:21.255602    2574 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9e42161-af91-4958-bebe-2a87ff299a1a-extensions-socket-volume\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:45:21.255647 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:21.255640    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpdwf\" (UniqueName: \"kubernetes.io/projected/f9e42161-af91-4958-bebe-2a87ff299a1a-kube-api-access-qpdwf\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:45:21.453948 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:21.453916    2574 generic.go:356] "Generic (PLEG): container finished" podID="f9e42161-af91-4958-bebe-2a87ff299a1a" containerID="4e5b6d9a8bb38d6f5fe068e7a0116fd4f715cc4a9cec30fdf3204db1ecfd3897" exitCode=0
Jul 17 09:45:21.454136 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:21.453966    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lmklg"
Jul 17 09:45:21.454136 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:21.454016    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lmklg" event={"ID":"f9e42161-af91-4958-bebe-2a87ff299a1a","Type":"ContainerDied","Data":"4e5b6d9a8bb38d6f5fe068e7a0116fd4f715cc4a9cec30fdf3204db1ecfd3897"}
Jul 17 09:45:21.454136 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:21.454076    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lmklg" event={"ID":"f9e42161-af91-4958-bebe-2a87ff299a1a","Type":"ContainerDied","Data":"1a585ba25d03ecefdf49c929b37a1b5fb6044ec303ec20608fc2953882634333"}
Jul 17 09:45:21.454136 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:21.454106    2574 scope.go:117] "RemoveContainer" containerID="4e5b6d9a8bb38d6f5fe068e7a0116fd4f715cc4a9cec30fdf3204db1ecfd3897"
Jul 17 09:45:21.467474 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:21.467455    2574 scope.go:117] "RemoveContainer" containerID="4e5b6d9a8bb38d6f5fe068e7a0116fd4f715cc4a9cec30fdf3204db1ecfd3897"
Jul 17 09:45:21.467751 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:45:21.467730    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e5b6d9a8bb38d6f5fe068e7a0116fd4f715cc4a9cec30fdf3204db1ecfd3897\": container with ID starting with 4e5b6d9a8bb38d6f5fe068e7a0116fd4f715cc4a9cec30fdf3204db1ecfd3897 not found: ID does not exist" containerID="4e5b6d9a8bb38d6f5fe068e7a0116fd4f715cc4a9cec30fdf3204db1ecfd3897"
Jul 17 09:45:21.467869 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:21.467758    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e5b6d9a8bb38d6f5fe068e7a0116fd4f715cc4a9cec30fdf3204db1ecfd3897"} err="failed to get container status \"4e5b6d9a8bb38d6f5fe068e7a0116fd4f715cc4a9cec30fdf3204db1ecfd3897\": rpc error: code = NotFound desc = could not find container \"4e5b6d9a8bb38d6f5fe068e7a0116fd4f715cc4a9cec30fdf3204db1ecfd3897\": container with ID starting with 4e5b6d9a8bb38d6f5fe068e7a0116fd4f715cc4a9cec30fdf3204db1ecfd3897 not found: ID does not exist"
Jul 17 09:45:21.479400 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:21.479376    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lmklg"]
Jul 17 09:45:21.482819 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:21.482796    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-lmklg"]
Jul 17 09:45:22.898374 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:45:22.898341    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9e42161-af91-4958-bebe-2a87ff299a1a" path="/var/lib/kubelet/pods/f9e42161-af91-4958-bebe-2a87ff299a1a/volumes"
Jul 17 09:46:18.901456 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:46:18.901419    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k8fhl"]
Jul 17 09:46:18.901931 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:46:18.901861    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9e42161-af91-4958-bebe-2a87ff299a1a" containerName="manager"
Jul 17 09:46:18.901931 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:46:18.901875    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9e42161-af91-4958-bebe-2a87ff299a1a" containerName="manager"
Jul 17 09:46:18.901931 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:46:18.901894    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1e47955-752c-4b2e-b76f-9d1efaa72250" containerName="cleanup"
Jul 17 09:46:18.901931 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:46:18.901900    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1e47955-752c-4b2e-b76f-9d1efaa72250" containerName="cleanup"
Jul 17 09:46:18.902068 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:46:18.901971    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9e42161-af91-4958-bebe-2a87ff299a1a" containerName="manager"
Jul 17 09:46:18.902068 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:46:18.901980    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1e47955-752c-4b2e-b76f-9d1efaa72250" containerName="cleanup"
Jul 17 09:46:18.905184 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:46:18.905168    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k8fhl"
Jul 17 09:46:18.908398 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:46:18.908382    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-h2vwj\""
Jul 17 09:46:18.916953 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:46:18.916928    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k8fhl"]
Jul 17 09:46:19.011510 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:46:19.011476    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tn45t\" (UniqueName: \"kubernetes.io/projected/bcd494e5-0002-47a1-9c6d-c31af2e510e2-kube-api-access-tn45t\") pod \"kuadrant-operator-controller-manager-86878c769f-k8fhl\" (UID: \"bcd494e5-0002-47a1-9c6d-c31af2e510e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k8fhl"
Jul 17 09:46:19.011698 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:46:19.011532    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcd494e5-0002-47a1-9c6d-c31af2e510e2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-k8fhl\" (UID: \"bcd494e5-0002-47a1-9c6d-c31af2e510e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k8fhl"
Jul 17 09:46:19.112550 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:46:19.112509    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn45t\" (UniqueName: \"kubernetes.io/projected/bcd494e5-0002-47a1-9c6d-c31af2e510e2-kube-api-access-tn45t\") pod \"kuadrant-operator-controller-manager-86878c769f-k8fhl\" (UID: \"bcd494e5-0002-47a1-9c6d-c31af2e510e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k8fhl"
Jul 17 09:46:19.112726 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:46:19.112572    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcd494e5-0002-47a1-9c6d-c31af2e510e2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-k8fhl\" (UID: \"bcd494e5-0002-47a1-9c6d-c31af2e510e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k8fhl"
Jul 17 09:46:19.112951 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:46:19.112929    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcd494e5-0002-47a1-9c6d-c31af2e510e2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-k8fhl\" (UID: \"bcd494e5-0002-47a1-9c6d-c31af2e510e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k8fhl"
Jul 17 09:46:19.121887 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:46:19.121864    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tn45t\" (UniqueName: \"kubernetes.io/projected/bcd494e5-0002-47a1-9c6d-c31af2e510e2-kube-api-access-tn45t\") pod \"kuadrant-operator-controller-manager-86878c769f-k8fhl\" (UID: \"bcd494e5-0002-47a1-9c6d-c31af2e510e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k8fhl"
Jul 17 09:46:19.215869 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:46:19.215830    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k8fhl"
Jul 17 09:46:19.366350 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:46:19.366323    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k8fhl"]
Jul 17 09:46:19.368450 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:46:19.368421    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcd494e5_0002_47a1_9c6d_c31af2e510e2.slice/crio-6f98f5d3ede0f1a7aa980f10eca29ad342d2387d268085cdf9c3a00870fac866 WatchSource:0}: Error finding container 6f98f5d3ede0f1a7aa980f10eca29ad342d2387d268085cdf9c3a00870fac866: Status 404 returned error can't find the container with id 6f98f5d3ede0f1a7aa980f10eca29ad342d2387d268085cdf9c3a00870fac866
Jul 17 09:46:19.761897 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:46:19.761861    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k8fhl" event={"ID":"bcd494e5-0002-47a1-9c6d-c31af2e510e2","Type":"ContainerStarted","Data":"4cf41f2278935bfe86754d63b61ca6a3e752e5c78201389ec638fbfcb70fca38"}
Jul 17 09:46:19.762080 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:46:19.761919    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k8fhl" event={"ID":"bcd494e5-0002-47a1-9c6d-c31af2e510e2","Type":"ContainerStarted","Data":"6f98f5d3ede0f1a7aa980f10eca29ad342d2387d268085cdf9c3a00870fac866"}
Jul 17 09:46:19.762080 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:46:19.761946    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k8fhl"
Jul 17 09:46:19.785338 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:46:19.785283    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k8fhl" podStartSLOduration=1.785264757 podStartE2EDuration="1.785264757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:46:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:46:19.782717776 +0000 UTC m=+1177.365356173" watchObservedRunningTime="2026-07-17 09:46:19.785264757 +0000 UTC m=+1177.367903134"
Jul 17 09:46:30.769585 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:46:30.769546    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-k8fhl"
Jul 17 09:46:42.930098 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:46:42.930067    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dh564_84db06ab-87e2-43b3-82d0-13f06ba86d06/ovn-acl-logging/0.log"
Jul 17 09:46:42.930878 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:46:42.930859    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dh564_84db06ab-87e2-43b3-82d0-13f06ba86d06/ovn-acl-logging/0.log"
Jul 17 09:51:42.967170 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:51:42.967141    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dh564_84db06ab-87e2-43b3-82d0-13f06ba86d06/ovn-acl-logging/0.log"
Jul 17 09:51:42.969807 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:51:42.969693    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dh564_84db06ab-87e2-43b3-82d0-13f06ba86d06/ovn-acl-logging/0.log"
Jul 17 09:52:27.324602 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:27.324565    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-bl7rh"]
Jul 17 09:52:27.328205 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:27.328188    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-bl7rh"
Jul 17 09:52:27.330959 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:27.330939    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 09:52:27.335175 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:27.335151    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-bl7rh"]
Jul 17 09:52:27.438173 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:27.438140    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c3a547d-a76b-43fc-b93e-fbfc684ff8b1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-bl7rh\" (UID: \"2c3a547d-a76b-43fc-b93e-fbfc684ff8b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-bl7rh"
Jul 17 09:52:27.539547 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:27.539511    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c3a547d-a76b-43fc-b93e-fbfc684ff8b1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-bl7rh\" (UID: \"2c3a547d-a76b-43fc-b93e-fbfc684ff8b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-bl7rh"
Jul 17 09:52:27.540160 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:27.540139    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c3a547d-a76b-43fc-b93e-fbfc684ff8b1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-bl7rh\" (UID: \"2c3a547d-a76b-43fc-b93e-fbfc684ff8b1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-bl7rh"
Jul 17 09:52:27.640807 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:27.640723    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-bl7rh"
Jul 17 09:52:27.787211 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:27.787183    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-bl7rh"]
Jul 17 09:52:27.788925 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:52:27.788889    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c3a547d_a76b_43fc_b93e_fbfc684ff8b1.slice/crio-30ad97c68a74068c6620c6e18c85b9a9ad2f7dd780f7b845e90b09e88c42c2ce WatchSource:0}: Error finding container 30ad97c68a74068c6620c6e18c85b9a9ad2f7dd780f7b845e90b09e88c42c2ce: Status 404 returned error can't find the container with id 30ad97c68a74068c6620c6e18c85b9a9ad2f7dd780f7b845e90b09e88c42c2ce
Jul 17 09:52:28.724378 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:28.724287    2574 generic.go:356] "Generic (PLEG): container finished" podID="2c3a547d-a76b-43fc-b93e-fbfc684ff8b1" containerID="da7b779ba73f03310c743fbb92201b766888d09db48b8ba64e05b32906c944d6" exitCode=7
Jul 17 09:52:28.724378 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:28.724326    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-bl7rh" event={"ID":"2c3a547d-a76b-43fc-b93e-fbfc684ff8b1","Type":"ContainerDied","Data":"da7b779ba73f03310c743fbb92201b766888d09db48b8ba64e05b32906c944d6"}
Jul 17 09:52:28.724378 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:28.724363    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-bl7rh" event={"ID":"2c3a547d-a76b-43fc-b93e-fbfc684ff8b1","Type":"ContainerStarted","Data":"30ad97c68a74068c6620c6e18c85b9a9ad2f7dd780f7b845e90b09e88c42c2ce"}
Jul 17 09:52:28.724786 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:28.724546    2574 scope.go:117] "RemoveContainer" containerID="da7b779ba73f03310c743fbb92201b766888d09db48b8ba64e05b32906c944d6"
Jul 17 09:52:29.731523 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:29.731474    2574 generic.go:356] "Generic (PLEG): container finished" podID="2c3a547d-a76b-43fc-b93e-fbfc684ff8b1" containerID="46b87f1f5f7b0d5b3004af9a05cd888b4df5930bd59259f5ea77dfe713a1873f" exitCode=7
Jul 17 09:52:29.731999 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:29.731555    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-bl7rh" event={"ID":"2c3a547d-a76b-43fc-b93e-fbfc684ff8b1","Type":"ContainerDied","Data":"46b87f1f5f7b0d5b3004af9a05cd888b4df5930bd59259f5ea77dfe713a1873f"}
Jul 17 09:52:29.731999 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:29.731615    2574 scope.go:117] "RemoveContainer" containerID="da7b779ba73f03310c743fbb92201b766888d09db48b8ba64e05b32906c944d6"
Jul 17 09:52:29.731999 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:29.731889    2574 scope.go:117] "RemoveContainer" containerID="46b87f1f5f7b0d5b3004af9a05cd888b4df5930bd59259f5ea77dfe713a1873f"
Jul 17 09:52:29.732209 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:52:29.732107    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-bl7rh_odh-ai-gateway-infra(2c3a547d-a76b-43fc-b93e-fbfc684ff8b1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-bl7rh" podUID="2c3a547d-a76b-43fc-b93e-fbfc684ff8b1"
Jul 17 09:52:30.739069 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:30.739040    2574 scope.go:117] "RemoveContainer" containerID="46b87f1f5f7b0d5b3004af9a05cd888b4df5930bd59259f5ea77dfe713a1873f"
Jul 17 09:52:30.739485 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:52:30.739315    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-bl7rh_odh-ai-gateway-infra(2c3a547d-a76b-43fc-b93e-fbfc684ff8b1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-bl7rh" podUID="2c3a547d-a76b-43fc-b93e-fbfc684ff8b1"
Jul 17 09:52:42.894014 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:42.893979    2574 scope.go:117] "RemoveContainer" containerID="46b87f1f5f7b0d5b3004af9a05cd888b4df5930bd59259f5ea77dfe713a1873f"
Jul 17 09:52:43.807788 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:43.807710    2574 generic.go:356] "Generic (PLEG): container finished" podID="2c3a547d-a76b-43fc-b93e-fbfc684ff8b1" containerID="0c2cf8022efa975d90e593ac4e71dee0ab9942a95194c1fabf6f08e37ab9d558" exitCode=7
Jul 17 09:52:43.807930 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:43.807783    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-bl7rh" event={"ID":"2c3a547d-a76b-43fc-b93e-fbfc684ff8b1","Type":"ContainerDied","Data":"0c2cf8022efa975d90e593ac4e71dee0ab9942a95194c1fabf6f08e37ab9d558"}
Jul 17 09:52:43.807930 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:43.807869    2574 scope.go:117] "RemoveContainer" containerID="46b87f1f5f7b0d5b3004af9a05cd888b4df5930bd59259f5ea77dfe713a1873f"
Jul 17 09:52:43.808265 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:43.808229    2574 scope.go:117] "RemoveContainer" containerID="0c2cf8022efa975d90e593ac4e71dee0ab9942a95194c1fabf6f08e37ab9d558"
Jul 17 09:52:43.808499 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:52:43.808481    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-bl7rh_odh-ai-gateway-infra(2c3a547d-a76b-43fc-b93e-fbfc684ff8b1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-bl7rh" podUID="2c3a547d-a76b-43fc-b93e-fbfc684ff8b1"
Jul 17 09:52:44.834821 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:44.834786    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-bl7rh"]
Jul 17 09:52:44.978016 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:44.977988    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-bl7rh"
Jul 17 09:52:45.002845 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:45.002815    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c3a547d-a76b-43fc-b93e-fbfc684ff8b1-maas-api-service-ca\") pod \"2c3a547d-a76b-43fc-b93e-fbfc684ff8b1\" (UID: \"2c3a547d-a76b-43fc-b93e-fbfc684ff8b1\") "
Jul 17 09:52:45.003124 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:45.003104    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c3a547d-a76b-43fc-b93e-fbfc684ff8b1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2c3a547d-a76b-43fc-b93e-fbfc684ff8b1" (UID: "2c3a547d-a76b-43fc-b93e-fbfc684ff8b1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:52:45.104422 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:45.104334    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c3a547d-a76b-43fc-b93e-fbfc684ff8b1-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:52:45.821744 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:45.821706    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-bl7rh"
Jul 17 09:52:45.821930 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:45.821751    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-bl7rh" event={"ID":"2c3a547d-a76b-43fc-b93e-fbfc684ff8b1","Type":"ContainerDied","Data":"30ad97c68a74068c6620c6e18c85b9a9ad2f7dd780f7b845e90b09e88c42c2ce"}
Jul 17 09:52:45.821930 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:45.821817    2574 scope.go:117] "RemoveContainer" containerID="0c2cf8022efa975d90e593ac4e71dee0ab9942a95194c1fabf6f08e37ab9d558"
Jul 17 09:52:45.849706 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:45.849682    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-bl7rh"]
Jul 17 09:52:45.851962 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:45.851943    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-bl7rh"]
Jul 17 09:52:46.898565 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:52:46.898529    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c3a547d-a76b-43fc-b93e-fbfc684ff8b1" path="/var/lib/kubelet/pods/2c3a547d-a76b-43fc-b93e-fbfc684ff8b1/volumes"
Jul 17 09:53:17.370139 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:17.370093    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rjk79"]
Jul 17 09:53:17.370743 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:17.370621    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c3a547d-a76b-43fc-b93e-fbfc684ff8b1" containerName="revoke-keys"
Jul 17 09:53:17.370743 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:17.370636    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c3a547d-a76b-43fc-b93e-fbfc684ff8b1" containerName="revoke-keys"
Jul 17 09:53:17.370743 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:17.370647    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c3a547d-a76b-43fc-b93e-fbfc684ff8b1" containerName="revoke-keys"
Jul 17 09:53:17.370743 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:17.370654    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c3a547d-a76b-43fc-b93e-fbfc684ff8b1" containerName="revoke-keys"
Jul 17 09:53:17.370743 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:17.370742    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c3a547d-a76b-43fc-b93e-fbfc684ff8b1" containerName="revoke-keys"
Jul 17 09:53:17.370984 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:17.370754    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c3a547d-a76b-43fc-b93e-fbfc684ff8b1" containerName="revoke-keys"
Jul 17 09:53:17.370984 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:17.370759    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c3a547d-a76b-43fc-b93e-fbfc684ff8b1" containerName="revoke-keys"
Jul 17 09:53:17.373806 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:17.373784    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rjk79"
Jul 17 09:53:17.376691 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:17.376668    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 09:53:17.380084 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:17.380058    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rjk79"]
Jul 17 09:53:17.513121 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:17.513076    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbe84a3f-865e-42a8-b725-768445ba4f1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-rjk79\" (UID: \"bbe84a3f-865e-42a8-b725-768445ba4f1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rjk79"
Jul 17 09:53:17.614148 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:17.614111    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbe84a3f-865e-42a8-b725-768445ba4f1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-rjk79\" (UID: \"bbe84a3f-865e-42a8-b725-768445ba4f1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rjk79"
Jul 17 09:53:17.614817 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:17.614788    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbe84a3f-865e-42a8-b725-768445ba4f1a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-rjk79\" (UID: \"bbe84a3f-865e-42a8-b725-768445ba4f1a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rjk79"
Jul 17 09:53:17.688081 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:17.688049    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rjk79"
Jul 17 09:53:18.030613 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:18.030591    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rjk79"]
Jul 17 09:53:18.033337 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:53:18.033307    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbe84a3f_865e_42a8_b725_768445ba4f1a.slice/crio-60f681c0836954adce4184eb81927e61a9eb7d0467121812ce34309f99dcd529 WatchSource:0}: Error finding container 60f681c0836954adce4184eb81927e61a9eb7d0467121812ce34309f99dcd529: Status 404 returned error can't find the container with id 60f681c0836954adce4184eb81927e61a9eb7d0467121812ce34309f99dcd529
Jul 17 09:53:18.997759 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:18.997722    2574 generic.go:356] "Generic (PLEG): container finished" podID="bbe84a3f-865e-42a8-b725-768445ba4f1a" containerID="7e1543155b2bf27ac83f17383ff0250819dbb837f6f3c4592eabc0426cc7450d" exitCode=7
Jul 17 09:53:18.998143 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:18.997810    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rjk79" event={"ID":"bbe84a3f-865e-42a8-b725-768445ba4f1a","Type":"ContainerDied","Data":"7e1543155b2bf27ac83f17383ff0250819dbb837f6f3c4592eabc0426cc7450d"}
Jul 17 09:53:18.998143 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:18.997859    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rjk79" event={"ID":"bbe84a3f-865e-42a8-b725-768445ba4f1a","Type":"ContainerStarted","Data":"60f681c0836954adce4184eb81927e61a9eb7d0467121812ce34309f99dcd529"}
Jul 17 09:53:18.998143 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:18.998040    2574 scope.go:117] "RemoveContainer" containerID="7e1543155b2bf27ac83f17383ff0250819dbb837f6f3c4592eabc0426cc7450d"
Jul 17 09:53:20.004905 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:20.004815    2574 generic.go:356] "Generic (PLEG): container finished" podID="bbe84a3f-865e-42a8-b725-768445ba4f1a" containerID="3fb28aaa17475e62facad4969d330b4c68d6798712ed458f70a26c8b5ac1cb66" exitCode=7
Jul 17 09:53:20.005297 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:20.004905    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rjk79" event={"ID":"bbe84a3f-865e-42a8-b725-768445ba4f1a","Type":"ContainerDied","Data":"3fb28aaa17475e62facad4969d330b4c68d6798712ed458f70a26c8b5ac1cb66"}
Jul 17 09:53:20.005297 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:20.004965    2574 scope.go:117] "RemoveContainer" containerID="7e1543155b2bf27ac83f17383ff0250819dbb837f6f3c4592eabc0426cc7450d"
Jul 17 09:53:20.005297 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:20.005138    2574 scope.go:117] "RemoveContainer" containerID="3fb28aaa17475e62facad4969d330b4c68d6798712ed458f70a26c8b5ac1cb66"
Jul 17 09:53:20.005436 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:53:20.005415    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-rjk79_odh-ai-gateway-infra(bbe84a3f-865e-42a8-b725-768445ba4f1a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rjk79" podUID="bbe84a3f-865e-42a8-b725-768445ba4f1a"
Jul 17 09:53:21.012028 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:21.012001    2574 scope.go:117] "RemoveContainer" containerID="3fb28aaa17475e62facad4969d330b4c68d6798712ed458f70a26c8b5ac1cb66"
Jul 17 09:53:21.012409 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:53:21.012260    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-rjk79_odh-ai-gateway-infra(bbe84a3f-865e-42a8-b725-768445ba4f1a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rjk79" podUID="bbe84a3f-865e-42a8-b725-768445ba4f1a"
Jul 17 09:53:32.895319 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:32.895285    2574 scope.go:117] "RemoveContainer" containerID="3fb28aaa17475e62facad4969d330b4c68d6798712ed458f70a26c8b5ac1cb66"
Jul 17 09:53:34.079428 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:34.079346    2574 generic.go:356] "Generic (PLEG): container finished" podID="bbe84a3f-865e-42a8-b725-768445ba4f1a" containerID="d8c05f4ee8c44767dc724afff5fafa34bd33c067fe73c903b127eaf011239941" exitCode=7
Jul 17 09:53:34.079812 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:34.079425    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rjk79" event={"ID":"bbe84a3f-865e-42a8-b725-768445ba4f1a","Type":"ContainerDied","Data":"d8c05f4ee8c44767dc724afff5fafa34bd33c067fe73c903b127eaf011239941"}
Jul 17 09:53:34.079812 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:34.079488    2574 scope.go:117] "RemoveContainer" containerID="3fb28aaa17475e62facad4969d330b4c68d6798712ed458f70a26c8b5ac1cb66"
Jul 17 09:53:34.079922 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:34.079835    2574 scope.go:117] "RemoveContainer" containerID="d8c05f4ee8c44767dc724afff5fafa34bd33c067fe73c903b127eaf011239941"
Jul 17 09:53:34.080084 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:53:34.080062    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-rjk79_odh-ai-gateway-infra(bbe84a3f-865e-42a8-b725-768445ba4f1a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rjk79" podUID="bbe84a3f-865e-42a8-b725-768445ba4f1a"
Jul 17 09:53:35.120981 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:35.120947    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rjk79"]
Jul 17 09:53:35.252759 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:35.252734    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rjk79"
Jul 17 09:53:35.281431 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:35.281405    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbe84a3f-865e-42a8-b725-768445ba4f1a-maas-api-service-ca\") pod \"bbe84a3f-865e-42a8-b725-768445ba4f1a\" (UID: \"bbe84a3f-865e-42a8-b725-768445ba4f1a\") "
Jul 17 09:53:35.281805 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:35.281769    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbe84a3f-865e-42a8-b725-768445ba4f1a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bbe84a3f-865e-42a8-b725-768445ba4f1a" (UID: "bbe84a3f-865e-42a8-b725-768445ba4f1a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:53:35.382578 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:35.382497    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbe84a3f-865e-42a8-b725-768445ba4f1a-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:53:36.092888 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:36.092850    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rjk79" event={"ID":"bbe84a3f-865e-42a8-b725-768445ba4f1a","Type":"ContainerDied","Data":"60f681c0836954adce4184eb81927e61a9eb7d0467121812ce34309f99dcd529"}
Jul 17 09:53:36.093086 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:36.092900    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rjk79"
Jul 17 09:53:36.093086 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:36.092936    2574 scope.go:117] "RemoveContainer" containerID="d8c05f4ee8c44767dc724afff5fafa34bd33c067fe73c903b127eaf011239941"
Jul 17 09:53:36.115699 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:36.115670    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rjk79"]
Jul 17 09:53:36.121118 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:36.121087    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rjk79"]
Jul 17 09:53:36.898264 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:53:36.898212    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbe84a3f-865e-42a8-b725-768445ba4f1a" path="/var/lib/kubelet/pods/bbe84a3f-865e-42a8-b725-768445ba4f1a/volumes"
Jul 17 09:54:07.416716 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:07.416679    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-k2qts"]
Jul 17 09:54:07.417408 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:07.417109    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbe84a3f-865e-42a8-b725-768445ba4f1a" containerName="revoke-keys"
Jul 17 09:54:07.417408 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:07.417121    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbe84a3f-865e-42a8-b725-768445ba4f1a" containerName="revoke-keys"
Jul 17 09:54:07.417408 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:07.417137    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbe84a3f-865e-42a8-b725-768445ba4f1a" containerName="revoke-keys"
Jul 17 09:54:07.417408 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:07.417143    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbe84a3f-865e-42a8-b725-768445ba4f1a" containerName="revoke-keys"
Jul 17 09:54:07.417408 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:07.417162    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c3a547d-a76b-43fc-b93e-fbfc684ff8b1" containerName="revoke-keys"
Jul 17 09:54:07.417408 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:07.417167    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c3a547d-a76b-43fc-b93e-fbfc684ff8b1" containerName="revoke-keys"
Jul 17 09:54:07.417408 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:07.417221    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbe84a3f-865e-42a8-b725-768445ba4f1a" containerName="revoke-keys"
Jul 17 09:54:07.417408 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:07.417231    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbe84a3f-865e-42a8-b725-768445ba4f1a" containerName="revoke-keys"
Jul 17 09:54:07.421848 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:07.421831    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-k2qts"
Jul 17 09:54:07.424696 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:07.424665    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 09:54:07.428068 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:07.428042    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-k2qts"]
Jul 17 09:54:07.492670 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:07.492636    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85f6e468-81b7-4e5e-b227-4d1179e6e23a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-k2qts\" (UID: \"85f6e468-81b7-4e5e-b227-4d1179e6e23a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-k2qts"
Jul 17 09:54:07.594109 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:07.594076    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85f6e468-81b7-4e5e-b227-4d1179e6e23a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-k2qts\" (UID: \"85f6e468-81b7-4e5e-b227-4d1179e6e23a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-k2qts"
Jul 17 09:54:07.594688 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:07.594668    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85f6e468-81b7-4e5e-b227-4d1179e6e23a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-k2qts\" (UID: \"85f6e468-81b7-4e5e-b227-4d1179e6e23a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-k2qts"
Jul 17 09:54:07.735163 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:07.735069    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-k2qts"
Jul 17 09:54:07.874760 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:07.874734    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-k2qts"]
Jul 17 09:54:07.876925 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:54:07.876882    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85f6e468_81b7_4e5e_b227_4d1179e6e23a.slice/crio-389f7fdd5e2946c81018ccb8f715368c96732c8cc35718f575821a2571e2b283 WatchSource:0}: Error finding container 389f7fdd5e2946c81018ccb8f715368c96732c8cc35718f575821a2571e2b283: Status 404 returned error can't find the container with id 389f7fdd5e2946c81018ccb8f715368c96732c8cc35718f575821a2571e2b283
Jul 17 09:54:08.267155 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:08.267118    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-k2qts" event={"ID":"85f6e468-81b7-4e5e-b227-4d1179e6e23a","Type":"ContainerStarted","Data":"c3ea9d40c5e3ab080d19c33e65fcb979509c7e43dd67e08385c774f13b2bbb62"}
Jul 17 09:54:08.267367 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:08.267184    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-k2qts" event={"ID":"85f6e468-81b7-4e5e-b227-4d1179e6e23a","Type":"ContainerStarted","Data":"389f7fdd5e2946c81018ccb8f715368c96732c8cc35718f575821a2571e2b283"}
Jul 17 09:54:08.283018 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:08.282971    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-k2qts" podStartSLOduration=1.282955875 podStartE2EDuration="1.282955875s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:54:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:54:08.281820548 +0000 UTC m=+1645.864458937" watchObservedRunningTime="2026-07-17 09:54:08.282955875 +0000 UTC m=+1645.865594249"
Jul 17 09:54:09.273955 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:09.273917    2574 generic.go:356] "Generic (PLEG): container finished" podID="85f6e468-81b7-4e5e-b227-4d1179e6e23a" containerID="c3ea9d40c5e3ab080d19c33e65fcb979509c7e43dd67e08385c774f13b2bbb62" exitCode=7
Jul 17 09:54:09.274466 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:09.274002    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-k2qts" event={"ID":"85f6e468-81b7-4e5e-b227-4d1179e6e23a","Type":"ContainerDied","Data":"c3ea9d40c5e3ab080d19c33e65fcb979509c7e43dd67e08385c774f13b2bbb62"}
Jul 17 09:54:09.274466 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:09.274282    2574 scope.go:117] "RemoveContainer" containerID="c3ea9d40c5e3ab080d19c33e65fcb979509c7e43dd67e08385c774f13b2bbb62"
Jul 17 09:54:10.280615 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:10.280577    2574 generic.go:356] "Generic (PLEG): container finished" podID="85f6e468-81b7-4e5e-b227-4d1179e6e23a" containerID="cc764488d7b84f777303b2ba518b4b6675b1adbed380b40b3b1195f604007a0a" exitCode=7
Jul 17 09:54:10.281061 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:10.280620    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-k2qts" event={"ID":"85f6e468-81b7-4e5e-b227-4d1179e6e23a","Type":"ContainerDied","Data":"cc764488d7b84f777303b2ba518b4b6675b1adbed380b40b3b1195f604007a0a"}
Jul 17 09:54:10.281061 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:10.280691    2574 scope.go:117] "RemoveContainer" containerID="c3ea9d40c5e3ab080d19c33e65fcb979509c7e43dd67e08385c774f13b2bbb62"
Jul 17 09:54:10.281061 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:10.280822    2574 scope.go:117] "RemoveContainer" containerID="cc764488d7b84f777303b2ba518b4b6675b1adbed380b40b3b1195f604007a0a"
Jul 17 09:54:10.281205 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:54:10.281078    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-k2qts_odh-ai-gateway-infra(85f6e468-81b7-4e5e-b227-4d1179e6e23a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-k2qts" podUID="85f6e468-81b7-4e5e-b227-4d1179e6e23a"
Jul 17 09:54:11.287974 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:11.287944    2574 scope.go:117] "RemoveContainer" containerID="cc764488d7b84f777303b2ba518b4b6675b1adbed380b40b3b1195f604007a0a"
Jul 17 09:54:11.288393 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:54:11.288156    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-k2qts_odh-ai-gateway-infra(85f6e468-81b7-4e5e-b227-4d1179e6e23a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-k2qts" podUID="85f6e468-81b7-4e5e-b227-4d1179e6e23a"
Jul 17 09:54:25.892356 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:25.892320    2574 scope.go:117] "RemoveContainer" containerID="cc764488d7b84f777303b2ba518b4b6675b1adbed380b40b3b1195f604007a0a"
Jul 17 09:54:26.369166 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:26.369125    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-k2qts" event={"ID":"85f6e468-81b7-4e5e-b227-4d1179e6e23a","Type":"ContainerStarted","Data":"193e3976a277266640b3da02eae01050de9fa6e3ed15b2ca554e659a5051f07e"}
Jul 17 09:54:27.376063 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:27.376025    2574 generic.go:356] "Generic (PLEG): container finished" podID="85f6e468-81b7-4e5e-b227-4d1179e6e23a" containerID="193e3976a277266640b3da02eae01050de9fa6e3ed15b2ca554e659a5051f07e" exitCode=7
Jul 17 09:54:27.376533 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:27.376097    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-k2qts" event={"ID":"85f6e468-81b7-4e5e-b227-4d1179e6e23a","Type":"ContainerDied","Data":"193e3976a277266640b3da02eae01050de9fa6e3ed15b2ca554e659a5051f07e"}
Jul 17 09:54:27.376533 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:27.376155    2574 scope.go:117] "RemoveContainer" containerID="cc764488d7b84f777303b2ba518b4b6675b1adbed380b40b3b1195f604007a0a"
Jul 17 09:54:27.376533 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:27.376459    2574 scope.go:117] "RemoveContainer" containerID="193e3976a277266640b3da02eae01050de9fa6e3ed15b2ca554e659a5051f07e"
Jul 17 09:54:27.376689 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:54:27.376665    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-k2qts_odh-ai-gateway-infra(85f6e468-81b7-4e5e-b227-4d1179e6e23a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-k2qts" podUID="85f6e468-81b7-4e5e-b227-4d1179e6e23a"
Jul 17 09:54:27.398885 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:27.398861    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-k2qts"]
Jul 17 09:54:28.524292 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:28.524267    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-k2qts"
Jul 17 09:54:28.703338 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:28.703306    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85f6e468-81b7-4e5e-b227-4d1179e6e23a-maas-api-service-ca\") pod \"85f6e468-81b7-4e5e-b227-4d1179e6e23a\" (UID: \"85f6e468-81b7-4e5e-b227-4d1179e6e23a\") "
Jul 17 09:54:28.703663 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:28.703642    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85f6e468-81b7-4e5e-b227-4d1179e6e23a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "85f6e468-81b7-4e5e-b227-4d1179e6e23a" (UID: "85f6e468-81b7-4e5e-b227-4d1179e6e23a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:54:28.804690 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:28.804658    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85f6e468-81b7-4e5e-b227-4d1179e6e23a-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:54:29.388784 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:29.388755    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-k2qts"
Jul 17 09:54:29.388784 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:29.388762    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-k2qts" event={"ID":"85f6e468-81b7-4e5e-b227-4d1179e6e23a","Type":"ContainerDied","Data":"389f7fdd5e2946c81018ccb8f715368c96732c8cc35718f575821a2571e2b283"}
Jul 17 09:54:29.389051 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:29.388821    2574 scope.go:117] "RemoveContainer" containerID="193e3976a277266640b3da02eae01050de9fa6e3ed15b2ca554e659a5051f07e"
Jul 17 09:54:29.408188 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:29.408162    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-k2qts"]
Jul 17 09:54:29.418668 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:29.418635    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-k2qts"]
Jul 17 09:54:30.897926 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:54:30.897891    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85f6e468-81b7-4e5e-b227-4d1179e6e23a" path="/var/lib/kubelet/pods/85f6e468-81b7-4e5e-b227-4d1179e6e23a/volumes"
Jul 17 09:55:07.465076 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:07.465001    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh"]
Jul 17 09:55:07.465508 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:07.465424    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85f6e468-81b7-4e5e-b227-4d1179e6e23a" containerName="revoke-keys"
Jul 17 09:55:07.465508 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:07.465435    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="85f6e468-81b7-4e5e-b227-4d1179e6e23a" containerName="revoke-keys"
Jul 17 09:55:07.465508 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:07.465457    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbe84a3f-865e-42a8-b725-768445ba4f1a" containerName="revoke-keys"
Jul 17 09:55:07.465508 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:07.465463    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbe84a3f-865e-42a8-b725-768445ba4f1a" containerName="revoke-keys"
Jul 17 09:55:07.465508 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:07.465475    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85f6e468-81b7-4e5e-b227-4d1179e6e23a" containerName="revoke-keys"
Jul 17 09:55:07.465508 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:07.465481    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="85f6e468-81b7-4e5e-b227-4d1179e6e23a" containerName="revoke-keys"
Jul 17 09:55:07.465714 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:07.465545    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="85f6e468-81b7-4e5e-b227-4d1179e6e23a" containerName="revoke-keys"
Jul 17 09:55:07.465714 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:07.465556    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="85f6e468-81b7-4e5e-b227-4d1179e6e23a" containerName="revoke-keys"
Jul 17 09:55:07.465714 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:07.465564    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbe84a3f-865e-42a8-b725-768445ba4f1a" containerName="revoke-keys"
Jul 17 09:55:07.468909 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:07.468889    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh"
Jul 17 09:55:07.471554 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:07.471537    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 09:55:07.476088 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:07.476061    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh"]
Jul 17 09:55:07.572488 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:07.572454    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1222971-04e5-4139-a5c7-d1e267aa1301-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh\" (UID: \"a1222971-04e5-4139-a5c7-d1e267aa1301\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh"
Jul 17 09:55:07.672986 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:07.672955    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1222971-04e5-4139-a5c7-d1e267aa1301-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh\" (UID: \"a1222971-04e5-4139-a5c7-d1e267aa1301\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh"
Jul 17 09:55:07.673535 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:07.673518    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1222971-04e5-4139-a5c7-d1e267aa1301-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh\" (UID: \"a1222971-04e5-4139-a5c7-d1e267aa1301\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh"
Jul 17 09:55:07.781147 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:07.781058    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh"
Jul 17 09:55:08.123733 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:08.123707    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh"]
Jul 17 09:55:08.126332 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:55:08.126305    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1222971_04e5_4139_a5c7_d1e267aa1301.slice/crio-e46d8863ffd4bcbd429b80f36bfa34d1d2c3b9aec74dd5ad0537fd98bdc4bb9f WatchSource:0}: Error finding container e46d8863ffd4bcbd429b80f36bfa34d1d2c3b9aec74dd5ad0537fd98bdc4bb9f: Status 404 returned error can't find the container with id e46d8863ffd4bcbd429b80f36bfa34d1d2c3b9aec74dd5ad0537fd98bdc4bb9f
Jul 17 09:55:08.605721 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:08.605684    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh" event={"ID":"a1222971-04e5-4139-a5c7-d1e267aa1301","Type":"ContainerStarted","Data":"ac4243b5d82cd4c1f70420bc688d2b08c30bef2a62debbad92f0d4ba56e310a2"}
Jul 17 09:55:08.606095 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:08.605772    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh" event={"ID":"a1222971-04e5-4139-a5c7-d1e267aa1301","Type":"ContainerStarted","Data":"e46d8863ffd4bcbd429b80f36bfa34d1d2c3b9aec74dd5ad0537fd98bdc4bb9f"}
Jul 17 09:55:08.625042 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:08.624997    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh" podStartSLOduration=1.624982851 podStartE2EDuration="1.624982851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:55:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:55:08.623925605 +0000 UTC m=+1706.206563980" watchObservedRunningTime="2026-07-17 09:55:08.624982851 +0000 UTC m=+1706.207621226"
Jul 17 09:55:09.613038 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:09.612995    2574 generic.go:356] "Generic (PLEG): container finished" podID="a1222971-04e5-4139-a5c7-d1e267aa1301" containerID="ac4243b5d82cd4c1f70420bc688d2b08c30bef2a62debbad92f0d4ba56e310a2" exitCode=7
Jul 17 09:55:09.613038 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:09.613057    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh" event={"ID":"a1222971-04e5-4139-a5c7-d1e267aa1301","Type":"ContainerDied","Data":"ac4243b5d82cd4c1f70420bc688d2b08c30bef2a62debbad92f0d4ba56e310a2"}
Jul 17 09:55:09.613568 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:09.613298    2574 scope.go:117] "RemoveContainer" containerID="ac4243b5d82cd4c1f70420bc688d2b08c30bef2a62debbad92f0d4ba56e310a2"
Jul 17 09:55:10.620798 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:10.620704    2574 generic.go:356] "Generic (PLEG): container finished" podID="a1222971-04e5-4139-a5c7-d1e267aa1301" containerID="88cf14e4c54616d5da1124b8373ede0ad56d0a034ad611470d88105cfb53b3fc" exitCode=7
Jul 17 09:55:10.621269 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:10.620790    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh" event={"ID":"a1222971-04e5-4139-a5c7-d1e267aa1301","Type":"ContainerDied","Data":"88cf14e4c54616d5da1124b8373ede0ad56d0a034ad611470d88105cfb53b3fc"}
Jul 17 09:55:10.621269 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:10.620849    2574 scope.go:117] "RemoveContainer" containerID="ac4243b5d82cd4c1f70420bc688d2b08c30bef2a62debbad92f0d4ba56e310a2"
Jul 17 09:55:10.621269 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:10.621105    2574 scope.go:117] "RemoveContainer" containerID="88cf14e4c54616d5da1124b8373ede0ad56d0a034ad611470d88105cfb53b3fc"
Jul 17 09:55:10.621440 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:55:10.621378    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh_odh-ai-gateway-infra(a1222971-04e5-4139-a5c7-d1e267aa1301)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh" podUID="a1222971-04e5-4139-a5c7-d1e267aa1301"
Jul 17 09:55:11.627604 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:11.627578    2574 scope.go:117] "RemoveContainer" containerID="88cf14e4c54616d5da1124b8373ede0ad56d0a034ad611470d88105cfb53b3fc"
Jul 17 09:55:11.627993 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:55:11.627761    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh_odh-ai-gateway-infra(a1222971-04e5-4139-a5c7-d1e267aa1301)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh" podUID="a1222971-04e5-4139-a5c7-d1e267aa1301"
Jul 17 09:55:25.891406 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:25.891366    2574 scope.go:117] "RemoveContainer" containerID="88cf14e4c54616d5da1124b8373ede0ad56d0a034ad611470d88105cfb53b3fc"
Jul 17 09:55:26.709173 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:26.709135    2574 generic.go:356] "Generic (PLEG): container finished" podID="a1222971-04e5-4139-a5c7-d1e267aa1301" containerID="52ac83f7642af1b191947d8ee8a08ccb56a050e8bcf116aa817ea4f740de1099" exitCode=7
Jul 17 09:55:26.709357 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:26.709197    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh" event={"ID":"a1222971-04e5-4139-a5c7-d1e267aa1301","Type":"ContainerDied","Data":"52ac83f7642af1b191947d8ee8a08ccb56a050e8bcf116aa817ea4f740de1099"}
Jul 17 09:55:26.709357 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:26.709274    2574 scope.go:117] "RemoveContainer" containerID="88cf14e4c54616d5da1124b8373ede0ad56d0a034ad611470d88105cfb53b3fc"
Jul 17 09:55:26.709595 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:26.709577    2574 scope.go:117] "RemoveContainer" containerID="52ac83f7642af1b191947d8ee8a08ccb56a050e8bcf116aa817ea4f740de1099"
Jul 17 09:55:26.709828 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:55:26.709810    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh_odh-ai-gateway-infra(a1222971-04e5-4139-a5c7-d1e267aa1301)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh" podUID="a1222971-04e5-4139-a5c7-d1e267aa1301"
Jul 17 09:55:27.734804 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:27.734772    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh"]
Jul 17 09:55:27.877884 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:27.877855    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh"
Jul 17 09:55:27.965014 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:27.964980    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1222971-04e5-4139-a5c7-d1e267aa1301-maas-api-service-ca\") pod \"a1222971-04e5-4139-a5c7-d1e267aa1301\" (UID: \"a1222971-04e5-4139-a5c7-d1e267aa1301\") "
Jul 17 09:55:27.965345 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:27.965323    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1222971-04e5-4139-a5c7-d1e267aa1301-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a1222971-04e5-4139-a5c7-d1e267aa1301" (UID: "a1222971-04e5-4139-a5c7-d1e267aa1301"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:55:28.066050 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:28.065961    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1222971-04e5-4139-a5c7-d1e267aa1301-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:55:28.210890 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:28.210857    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6c5c6b98f-rlrg6_8ab844e0-7c94-4003-9e42-ada02b05d4e9/manager/0.log"
Jul 17 09:55:28.722837 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:28.722807    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh"
Jul 17 09:55:28.723023 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:28.722838    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh" event={"ID":"a1222971-04e5-4139-a5c7-d1e267aa1301","Type":"ContainerDied","Data":"e46d8863ffd4bcbd429b80f36bfa34d1d2c3b9aec74dd5ad0537fd98bdc4bb9f"}
Jul 17 09:55:28.723023 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:28.722892    2574 scope.go:117] "RemoveContainer" containerID="52ac83f7642af1b191947d8ee8a08ccb56a050e8bcf116aa817ea4f740de1099"
Jul 17 09:55:28.746951 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:28.746924    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh"]
Jul 17 09:55:28.759511 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:28.758431    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9q4lh"]
Jul 17 09:55:28.898040 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:28.898005    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1222971-04e5-4139-a5c7-d1e267aa1301" path="/var/lib/kubelet/pods/a1222971-04e5-4139-a5c7-d1e267aa1301/volumes"
Jul 17 09:55:46.302458 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:46.302417    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-vzspl"]
Jul 17 09:55:46.303032 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:46.303013    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1222971-04e5-4139-a5c7-d1e267aa1301" containerName="revoke-keys"
Jul 17 09:55:46.303032 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:46.303033    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1222971-04e5-4139-a5c7-d1e267aa1301" containerName="revoke-keys"
Jul 17 09:55:46.303170 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:46.303044    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1222971-04e5-4139-a5c7-d1e267aa1301" containerName="revoke-keys"
Jul 17 09:55:46.303170 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:46.303050    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1222971-04e5-4139-a5c7-d1e267aa1301" containerName="revoke-keys"
Jul 17 09:55:46.303170 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:46.303068    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1222971-04e5-4139-a5c7-d1e267aa1301" containerName="revoke-keys"
Jul 17 09:55:46.303170 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:46.303073    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1222971-04e5-4139-a5c7-d1e267aa1301" containerName="revoke-keys"
Jul 17 09:55:46.303170 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:46.303079    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85f6e468-81b7-4e5e-b227-4d1179e6e23a" containerName="revoke-keys"
Jul 17 09:55:46.303170 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:46.303083    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="85f6e468-81b7-4e5e-b227-4d1179e6e23a" containerName="revoke-keys"
Jul 17 09:55:46.303170 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:46.303151    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1222971-04e5-4139-a5c7-d1e267aa1301" containerName="revoke-keys"
Jul 17 09:55:46.303170 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:46.303159    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="85f6e468-81b7-4e5e-b227-4d1179e6e23a" containerName="revoke-keys"
Jul 17 09:55:46.303170 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:46.303165    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1222971-04e5-4139-a5c7-d1e267aa1301" containerName="revoke-keys"
Jul 17 09:55:46.306279 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:46.306256    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-vzspl"
Jul 17 09:55:46.308861 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:46.308839    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 09:55:46.312818 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:46.312790    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-vzspl"]
Jul 17 09:55:46.454634 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:46.454599    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e9678bd-2b09-4a05-9ee9-68fc15a21514-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-vzspl\" (UID: \"3e9678bd-2b09-4a05-9ee9-68fc15a21514\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-vzspl"
Jul 17 09:55:46.555780 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:46.555702    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e9678bd-2b09-4a05-9ee9-68fc15a21514-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-vzspl\" (UID: \"3e9678bd-2b09-4a05-9ee9-68fc15a21514\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-vzspl"
Jul 17 09:55:46.556299 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:46.556279    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e9678bd-2b09-4a05-9ee9-68fc15a21514-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-vzspl\" (UID: \"3e9678bd-2b09-4a05-9ee9-68fc15a21514\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-vzspl"
Jul 17 09:55:46.620330 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:46.620296    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-vzspl"
Jul 17 09:55:46.765991 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:46.765956    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-vzspl"]
Jul 17 09:55:46.767817 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:55:46.767788    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e9678bd_2b09_4a05_9ee9_68fc15a21514.slice/crio-4eca42dce6b239f71fff556dea8f47a63e57b083668114a8e8723f1b626e35ab WatchSource:0}: Error finding container 4eca42dce6b239f71fff556dea8f47a63e57b083668114a8e8723f1b626e35ab: Status 404 returned error can't find the container with id 4eca42dce6b239f71fff556dea8f47a63e57b083668114a8e8723f1b626e35ab
Jul 17 09:55:46.826089 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:46.826051    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-vzspl" event={"ID":"3e9678bd-2b09-4a05-9ee9-68fc15a21514","Type":"ContainerStarted","Data":"4eca42dce6b239f71fff556dea8f47a63e57b083668114a8e8723f1b626e35ab"}
Jul 17 09:55:47.833134 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:47.833056    2574 generic.go:356] "Generic (PLEG): container finished" podID="3e9678bd-2b09-4a05-9ee9-68fc15a21514" containerID="f210f8d4bb4942864399a405bb642ba229d8a422a853bd6e6a697348ad041100" exitCode=7
Jul 17 09:55:47.833505 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:47.833143    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-vzspl" event={"ID":"3e9678bd-2b09-4a05-9ee9-68fc15a21514","Type":"ContainerDied","Data":"f210f8d4bb4942864399a405bb642ba229d8a422a853bd6e6a697348ad041100"}
Jul 17 09:55:47.833505 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:47.833329    2574 scope.go:117] "RemoveContainer" containerID="f210f8d4bb4942864399a405bb642ba229d8a422a853bd6e6a697348ad041100"
Jul 17 09:55:48.841694 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:48.841653    2574 generic.go:356] "Generic (PLEG): container finished" podID="3e9678bd-2b09-4a05-9ee9-68fc15a21514" containerID="d7778057c05305bbf3bfe25ef1ed102ae3ae83c7aee0c7415735fc3da30417d5" exitCode=7
Jul 17 09:55:48.842097 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:48.841695    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-vzspl" event={"ID":"3e9678bd-2b09-4a05-9ee9-68fc15a21514","Type":"ContainerDied","Data":"d7778057c05305bbf3bfe25ef1ed102ae3ae83c7aee0c7415735fc3da30417d5"}
Jul 17 09:55:48.842097 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:48.841803    2574 scope.go:117] "RemoveContainer" containerID="f210f8d4bb4942864399a405bb642ba229d8a422a853bd6e6a697348ad041100"
Jul 17 09:55:48.842097 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:48.841960    2574 scope.go:117] "RemoveContainer" containerID="d7778057c05305bbf3bfe25ef1ed102ae3ae83c7aee0c7415735fc3da30417d5"
Jul 17 09:55:48.842292 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:55:48.842214    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-vzspl_odh-ai-gateway-infra(3e9678bd-2b09-4a05-9ee9-68fc15a21514)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-vzspl" podUID="3e9678bd-2b09-4a05-9ee9-68fc15a21514"
Jul 17 09:55:49.849637 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:55:49.849608    2574 scope.go:117] "RemoveContainer" containerID="d7778057c05305bbf3bfe25ef1ed102ae3ae83c7aee0c7415735fc3da30417d5"
Jul 17 09:55:49.850043 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:55:49.849815    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-vzspl_odh-ai-gateway-infra(3e9678bd-2b09-4a05-9ee9-68fc15a21514)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-vzspl" podUID="3e9678bd-2b09-4a05-9ee9-68fc15a21514"
Jul 17 09:56:04.891433 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:04.891397    2574 scope.go:117] "RemoveContainer" containerID="d7778057c05305bbf3bfe25ef1ed102ae3ae83c7aee0c7415735fc3da30417d5"
Jul 17 09:56:05.934106 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:05.934073    2574 generic.go:356] "Generic (PLEG): container finished" podID="3e9678bd-2b09-4a05-9ee9-68fc15a21514" containerID="52a3bdb6fed1682fe420f2f568f4003fb70e3edb7b2ed4b9872d4b6b712e5fa4" exitCode=7
Jul 17 09:56:05.934552 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:05.934144    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-vzspl" event={"ID":"3e9678bd-2b09-4a05-9ee9-68fc15a21514","Type":"ContainerDied","Data":"52a3bdb6fed1682fe420f2f568f4003fb70e3edb7b2ed4b9872d4b6b712e5fa4"}
Jul 17 09:56:05.934552 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:05.934200    2574 scope.go:117] "RemoveContainer" containerID="d7778057c05305bbf3bfe25ef1ed102ae3ae83c7aee0c7415735fc3da30417d5"
Jul 17 09:56:05.934552 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:05.934481    2574 scope.go:117] "RemoveContainer" containerID="52a3bdb6fed1682fe420f2f568f4003fb70e3edb7b2ed4b9872d4b6b712e5fa4"
Jul 17 09:56:05.934716 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:56:05.934691    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-vzspl_odh-ai-gateway-infra(3e9678bd-2b09-4a05-9ee9-68fc15a21514)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-vzspl" podUID="3e9678bd-2b09-4a05-9ee9-68fc15a21514"
Jul 17 09:56:06.962548 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:06.962516    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-vzspl"]
Jul 17 09:56:07.106265 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:07.106212    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-vzspl"
Jul 17 09:56:07.142525 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:07.142490    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e9678bd-2b09-4a05-9ee9-68fc15a21514-maas-api-service-ca\") pod \"3e9678bd-2b09-4a05-9ee9-68fc15a21514\" (UID: \"3e9678bd-2b09-4a05-9ee9-68fc15a21514\") "
Jul 17 09:56:07.142877 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:07.142855    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e9678bd-2b09-4a05-9ee9-68fc15a21514-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3e9678bd-2b09-4a05-9ee9-68fc15a21514" (UID: "3e9678bd-2b09-4a05-9ee9-68fc15a21514"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:56:07.244161 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:07.244074    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e9678bd-2b09-4a05-9ee9-68fc15a21514-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:56:07.517676 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:07.517584    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-spwdr"]
Jul 17 09:56:07.518135 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:07.518114    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e9678bd-2b09-4a05-9ee9-68fc15a21514" containerName="revoke-keys"
Jul 17 09:56:07.518231 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:07.518138    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e9678bd-2b09-4a05-9ee9-68fc15a21514" containerName="revoke-keys"
Jul 17 09:56:07.518231 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:07.518154    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e9678bd-2b09-4a05-9ee9-68fc15a21514" containerName="revoke-keys"
Jul 17 09:56:07.518231 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:07.518161    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e9678bd-2b09-4a05-9ee9-68fc15a21514" containerName="revoke-keys"
Jul 17 09:56:07.518421 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:07.518314    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1222971-04e5-4139-a5c7-d1e267aa1301" containerName="revoke-keys"
Jul 17 09:56:07.518421 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:07.518332    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e9678bd-2b09-4a05-9ee9-68fc15a21514" containerName="revoke-keys"
Jul 17 09:56:07.518421 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:07.518346    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e9678bd-2b09-4a05-9ee9-68fc15a21514" containerName="revoke-keys"
Jul 17 09:56:07.521625 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:07.521602    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-spwdr"
Jul 17 09:56:07.530157 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:07.530135    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-spwdr"]
Jul 17 09:56:07.647263 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:07.647217    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67c3c50a-248a-4c13-98b0-aee7d41278e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-spwdr\" (UID: \"67c3c50a-248a-4c13-98b0-aee7d41278e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-spwdr"
Jul 17 09:56:07.748693 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:07.748661    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67c3c50a-248a-4c13-98b0-aee7d41278e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-spwdr\" (UID: \"67c3c50a-248a-4c13-98b0-aee7d41278e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-spwdr"
Jul 17 09:56:07.749346 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:07.749321    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67c3c50a-248a-4c13-98b0-aee7d41278e0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-spwdr\" (UID: \"67c3c50a-248a-4c13-98b0-aee7d41278e0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-spwdr"
Jul 17 09:56:07.832790 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:07.832717    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-spwdr"
Jul 17 09:56:07.950110 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:07.949954    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-vzspl"
Jul 17 09:56:07.950110 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:07.949972    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-vzspl" event={"ID":"3e9678bd-2b09-4a05-9ee9-68fc15a21514","Type":"ContainerDied","Data":"4eca42dce6b239f71fff556dea8f47a63e57b083668114a8e8723f1b626e35ab"}
Jul 17 09:56:07.950110 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:07.950035    2574 scope.go:117] "RemoveContainer" containerID="52a3bdb6fed1682fe420f2f568f4003fb70e3edb7b2ed4b9872d4b6b712e5fa4"
Jul 17 09:56:07.977270 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:07.977229    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-vzspl"]
Jul 17 09:56:07.985970 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:07.985948    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-vzspl"]
Jul 17 09:56:08.186435 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:08.186410    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-spwdr"]
Jul 17 09:56:08.188766 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:56:08.188738    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67c3c50a_248a_4c13_98b0_aee7d41278e0.slice/crio-b00bdef06e0bb5cae90cb075d6556d753495afc9e33b1d73072fc5f500809eaf WatchSource:0}: Error finding container b00bdef06e0bb5cae90cb075d6556d753495afc9e33b1d73072fc5f500809eaf: Status 404 returned error can't find the container with id b00bdef06e0bb5cae90cb075d6556d753495afc9e33b1d73072fc5f500809eaf
Jul 17 09:56:08.898348 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:08.898318    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e9678bd-2b09-4a05-9ee9-68fc15a21514" path="/var/lib/kubelet/pods/3e9678bd-2b09-4a05-9ee9-68fc15a21514/volumes"
Jul 17 09:56:08.958286 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:08.958208    2574 generic.go:356] "Generic (PLEG): container finished" podID="67c3c50a-248a-4c13-98b0-aee7d41278e0" containerID="25cde1db2e87d377b78445be229b5bab0cbd0564230e1ccb140f4c14ecf955dd" exitCode=6
Jul 17 09:56:08.958286 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:08.958272    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-spwdr" event={"ID":"67c3c50a-248a-4c13-98b0-aee7d41278e0","Type":"ContainerDied","Data":"25cde1db2e87d377b78445be229b5bab0cbd0564230e1ccb140f4c14ecf955dd"}
Jul 17 09:56:08.958492 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:08.958360    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-spwdr" event={"ID":"67c3c50a-248a-4c13-98b0-aee7d41278e0","Type":"ContainerStarted","Data":"b00bdef06e0bb5cae90cb075d6556d753495afc9e33b1d73072fc5f500809eaf"}
Jul 17 09:56:08.958492 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:08.958464    2574 scope.go:117] "RemoveContainer" containerID="25cde1db2e87d377b78445be229b5bab0cbd0564230e1ccb140f4c14ecf955dd"
Jul 17 09:56:09.965594 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:09.965556    2574 generic.go:356] "Generic (PLEG): container finished" podID="67c3c50a-248a-4c13-98b0-aee7d41278e0" containerID="4da7f82df37c6d2442d5a834ae575cea9297c2cfeca53e758db65ae3be23a768" exitCode=6
Jul 17 09:56:09.966085 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:09.965643    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-spwdr" event={"ID":"67c3c50a-248a-4c13-98b0-aee7d41278e0","Type":"ContainerDied","Data":"4da7f82df37c6d2442d5a834ae575cea9297c2cfeca53e758db65ae3be23a768"}
Jul 17 09:56:09.966085 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:09.965703    2574 scope.go:117] "RemoveContainer" containerID="25cde1db2e87d377b78445be229b5bab0cbd0564230e1ccb140f4c14ecf955dd"
Jul 17 09:56:09.966085 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:09.965925    2574 scope.go:117] "RemoveContainer" containerID="4da7f82df37c6d2442d5a834ae575cea9297c2cfeca53e758db65ae3be23a768"
Jul 17 09:56:09.966278 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:56:09.966219    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-spwdr_odh-ai-gateway-infra(67c3c50a-248a-4c13-98b0-aee7d41278e0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-spwdr" podUID="67c3c50a-248a-4c13-98b0-aee7d41278e0"
Jul 17 09:56:10.973108 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:10.973076    2574 scope.go:117] "RemoveContainer" containerID="4da7f82df37c6d2442d5a834ae575cea9297c2cfeca53e758db65ae3be23a768"
Jul 17 09:56:10.973537 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:56:10.973296    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-spwdr_odh-ai-gateway-infra(67c3c50a-248a-4c13-98b0-aee7d41278e0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-spwdr" podUID="67c3c50a-248a-4c13-98b0-aee7d41278e0"
Jul 17 09:56:24.891812 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:24.891777    2574 scope.go:117] "RemoveContainer" containerID="4da7f82df37c6d2442d5a834ae575cea9297c2cfeca53e758db65ae3be23a768"
Jul 17 09:56:26.051178 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:26.051143    2574 generic.go:356] "Generic (PLEG): container finished" podID="67c3c50a-248a-4c13-98b0-aee7d41278e0" containerID="ff387fa5bba715c8b7082dbdad20e0a614c8f11afab5606eed36000263823615" exitCode=6
Jul 17 09:56:26.051671 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:26.051219    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-spwdr" event={"ID":"67c3c50a-248a-4c13-98b0-aee7d41278e0","Type":"ContainerDied","Data":"ff387fa5bba715c8b7082dbdad20e0a614c8f11afab5606eed36000263823615"}
Jul 17 09:56:26.051671 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:26.051310    2574 scope.go:117] "RemoveContainer" containerID="4da7f82df37c6d2442d5a834ae575cea9297c2cfeca53e758db65ae3be23a768"
Jul 17 09:56:26.051671 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:26.051562    2574 scope.go:117] "RemoveContainer" containerID="ff387fa5bba715c8b7082dbdad20e0a614c8f11afab5606eed36000263823615"
Jul 17 09:56:26.051811 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:56:26.051787    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-spwdr_odh-ai-gateway-infra(67c3c50a-248a-4c13-98b0-aee7d41278e0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-spwdr" podUID="67c3c50a-248a-4c13-98b0-aee7d41278e0"
Jul 17 09:56:27.076965 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:27.076926    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-spwdr"]
Jul 17 09:56:27.223296 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:27.223271    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-spwdr"
Jul 17 09:56:27.348287 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:27.348178    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67c3c50a-248a-4c13-98b0-aee7d41278e0-maas-api-service-ca\") pod \"67c3c50a-248a-4c13-98b0-aee7d41278e0\" (UID: \"67c3c50a-248a-4c13-98b0-aee7d41278e0\") "
Jul 17 09:56:27.348535 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:27.348515    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67c3c50a-248a-4c13-98b0-aee7d41278e0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "67c3c50a-248a-4c13-98b0-aee7d41278e0" (UID: "67c3c50a-248a-4c13-98b0-aee7d41278e0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:56:27.449246 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:27.449216    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/67c3c50a-248a-4c13-98b0-aee7d41278e0-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:56:28.064557 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:28.064519    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-spwdr" event={"ID":"67c3c50a-248a-4c13-98b0-aee7d41278e0","Type":"ContainerDied","Data":"b00bdef06e0bb5cae90cb075d6556d753495afc9e33b1d73072fc5f500809eaf"}
Jul 17 09:56:28.064557 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:28.064544    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-spwdr"
Jul 17 09:56:28.064779 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:28.064611    2574 scope.go:117] "RemoveContainer" containerID="ff387fa5bba715c8b7082dbdad20e0a614c8f11afab5606eed36000263823615"
Jul 17 09:56:28.090493 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:28.090462    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-spwdr"]
Jul 17 09:56:28.095003 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:28.094972    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-spwdr"]
Jul 17 09:56:28.897583 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:28.897550    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67c3c50a-248a-4c13-98b0-aee7d41278e0" path="/var/lib/kubelet/pods/67c3c50a-248a-4c13-98b0-aee7d41278e0/volumes"
Jul 17 09:56:43.001917 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:43.001824    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dh564_84db06ab-87e2-43b3-82d0-13f06ba86d06/ovn-acl-logging/0.log"
Jul 17 09:56:43.008573 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:43.006285    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dh564_84db06ab-87e2-43b3-82d0-13f06ba86d06/ovn-acl-logging/0.log"
Jul 17 09:56:46.367638 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:46.367598    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-gr82w"]
Jul 17 09:56:46.368262 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:46.368227    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67c3c50a-248a-4c13-98b0-aee7d41278e0" containerName="revoke-keys"
Jul 17 09:56:46.368262 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:46.368264    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="67c3c50a-248a-4c13-98b0-aee7d41278e0" containerName="revoke-keys"
Jul 17 09:56:46.368435 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:46.368277    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67c3c50a-248a-4c13-98b0-aee7d41278e0" containerName="revoke-keys"
Jul 17 09:56:46.368435 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:46.368285    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="67c3c50a-248a-4c13-98b0-aee7d41278e0" containerName="revoke-keys"
Jul 17 09:56:46.368435 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:46.368293    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67c3c50a-248a-4c13-98b0-aee7d41278e0" containerName="revoke-keys"
Jul 17 09:56:46.368435 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:46.368301    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="67c3c50a-248a-4c13-98b0-aee7d41278e0" containerName="revoke-keys"
Jul 17 09:56:46.368435 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:46.368324    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e9678bd-2b09-4a05-9ee9-68fc15a21514" containerName="revoke-keys"
Jul 17 09:56:46.368435 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:46.368332    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e9678bd-2b09-4a05-9ee9-68fc15a21514" containerName="revoke-keys"
Jul 17 09:56:46.368435 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:46.368430    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e9678bd-2b09-4a05-9ee9-68fc15a21514" containerName="revoke-keys"
Jul 17 09:56:46.368768 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:46.368444    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="67c3c50a-248a-4c13-98b0-aee7d41278e0" containerName="revoke-keys"
Jul 17 09:56:46.368768 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:46.368453    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="67c3c50a-248a-4c13-98b0-aee7d41278e0" containerName="revoke-keys"
Jul 17 09:56:46.368768 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:46.368466    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="67c3c50a-248a-4c13-98b0-aee7d41278e0" containerName="revoke-keys"
Jul 17 09:56:46.371556 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:46.371536    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-gr82w"
Jul 17 09:56:46.374316 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:46.374295    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 09:56:46.378270 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:46.378226    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-gr82w"]
Jul 17 09:56:46.409133 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:46.409110    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-gr82w\" (UID: \"50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-gr82w"
Jul 17 09:56:46.510607 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:46.510582    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-gr82w\" (UID: \"50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-gr82w"
Jul 17 09:56:46.511130 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:46.511112    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-gr82w\" (UID: \"50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-gr82w"
Jul 17 09:56:46.683558 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:46.683538    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-gr82w"
Jul 17 09:56:46.815292 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:46.815269    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-gr82w"]
Jul 17 09:56:46.818428 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:56:46.818391    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50ace1ee_a9f1_4407_9e4d_36a6f7a69bd0.slice/crio-e22dfe4a7581447e791ab00a2de87d2e077e871475c1a0696542edca6a97ea12 WatchSource:0}: Error finding container e22dfe4a7581447e791ab00a2de87d2e077e871475c1a0696542edca6a97ea12: Status 404 returned error can't find the container with id e22dfe4a7581447e791ab00a2de87d2e077e871475c1a0696542edca6a97ea12
Jul 17 09:56:47.162102 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:47.162069    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-gr82w" event={"ID":"50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0","Type":"ContainerStarted","Data":"3587c533428d8c44d44666934a901a67428d4391e274d86eacbe8569d39834e1"}
Jul 17 09:56:47.162481 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:47.162132    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-gr82w" event={"ID":"50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0","Type":"ContainerStarted","Data":"e22dfe4a7581447e791ab00a2de87d2e077e871475c1a0696542edca6a97ea12"}
Jul 17 09:56:47.180162 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:47.180123    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-gr82w" podStartSLOduration=1.18011064 podStartE2EDuration="1.18011064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:56:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:56:47.178460573 +0000 UTC m=+1804.761098948" watchObservedRunningTime="2026-07-17 09:56:47.18011064 +0000 UTC m=+1804.762749014"
Jul 17 09:56:47.185192 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:47.185151    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6c5c6b98f-rlrg6_8ab844e0-7c94-4003-9e42-ada02b05d4e9/manager/0.log"
Jul 17 09:56:48.170759 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:48.170716    2574 generic.go:356] "Generic (PLEG): container finished" podID="50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0" containerID="3587c533428d8c44d44666934a901a67428d4391e274d86eacbe8569d39834e1" exitCode=7
Jul 17 09:56:48.171208 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:48.170901    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-gr82w" event={"ID":"50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0","Type":"ContainerDied","Data":"3587c533428d8c44d44666934a901a67428d4391e274d86eacbe8569d39834e1"}
Jul 17 09:56:48.171208 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:48.171183    2574 scope.go:117] "RemoveContainer" containerID="3587c533428d8c44d44666934a901a67428d4391e274d86eacbe8569d39834e1"
Jul 17 09:56:49.177139 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:49.177110    2574 generic.go:356] "Generic (PLEG): container finished" podID="50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0" containerID="21357b632ed035fa3d99a88281fc970e5aa5cd4ae6b6caab6de03c8fb10fe022" exitCode=7
Jul 17 09:56:49.177512 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:49.177193    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-gr82w" event={"ID":"50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0","Type":"ContainerDied","Data":"21357b632ed035fa3d99a88281fc970e5aa5cd4ae6b6caab6de03c8fb10fe022"}
Jul 17 09:56:49.177512 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:49.177271    2574 scope.go:117] "RemoveContainer" containerID="3587c533428d8c44d44666934a901a67428d4391e274d86eacbe8569d39834e1"
Jul 17 09:56:49.177512 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:49.177351    2574 scope.go:117] "RemoveContainer" containerID="21357b632ed035fa3d99a88281fc970e5aa5cd4ae6b6caab6de03c8fb10fe022"
Jul 17 09:56:49.177677 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:56:49.177534    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-gr82w_odh-ai-gateway-infra(50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-gr82w" podUID="50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0"
Jul 17 09:56:50.184332 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:56:50.184305    2574 scope.go:117] "RemoveContainer" containerID="21357b632ed035fa3d99a88281fc970e5aa5cd4ae6b6caab6de03c8fb10fe022"
Jul 17 09:56:50.184675 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:56:50.184460    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-gr82w_odh-ai-gateway-infra(50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-gr82w" podUID="50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0"
Jul 17 09:57:02.895486 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:02.895391    2574 scope.go:117] "RemoveContainer" containerID="21357b632ed035fa3d99a88281fc970e5aa5cd4ae6b6caab6de03c8fb10fe022"
Jul 17 09:57:03.255085 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:03.255055    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-gr82w" event={"ID":"50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0","Type":"ContainerStarted","Data":"ef7c31da0613e517aaf94cc85e49afa9b7237eb3ccf7891549a89126626089fa"}
Jul 17 09:57:04.262090 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:04.262056    2574 generic.go:356] "Generic (PLEG): container finished" podID="50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0" containerID="ef7c31da0613e517aaf94cc85e49afa9b7237eb3ccf7891549a89126626089fa" exitCode=6
Jul 17 09:57:04.262556 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:04.262110    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-gr82w" event={"ID":"50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0","Type":"ContainerDied","Data":"ef7c31da0613e517aaf94cc85e49afa9b7237eb3ccf7891549a89126626089fa"}
Jul 17 09:57:04.262556 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:04.262158    2574 scope.go:117] "RemoveContainer" containerID="21357b632ed035fa3d99a88281fc970e5aa5cd4ae6b6caab6de03c8fb10fe022"
Jul 17 09:57:04.262556 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:04.262471    2574 scope.go:117] "RemoveContainer" containerID="ef7c31da0613e517aaf94cc85e49afa9b7237eb3ccf7891549a89126626089fa"
Jul 17 09:57:04.262718 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:57:04.262685    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-gr82w_odh-ai-gateway-infra(50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-gr82w" podUID="50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0"
Jul 17 09:57:04.280433 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:04.280411    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-gr82w"]
Jul 17 09:57:05.408080 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:05.408051    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-gr82w"
Jul 17 09:57:05.456122 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:05.456090    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp"]
Jul 17 09:57:05.456791 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:05.456768    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0" containerName="revoke-keys"
Jul 17 09:57:05.456874 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:05.456796    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0" containerName="revoke-keys"
Jul 17 09:57:05.456874 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:05.456819    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0" containerName="revoke-keys"
Jul 17 09:57:05.456874 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:05.456828    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0" containerName="revoke-keys"
Jul 17 09:57:05.456874 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:05.456843    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0" containerName="revoke-keys"
Jul 17 09:57:05.456874 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:05.456852    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0" containerName="revoke-keys"
Jul 17 09:57:05.457119 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:05.456954    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0" containerName="revoke-keys"
Jul 17 09:57:05.457119 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:05.456969    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0" containerName="revoke-keys"
Jul 17 09:57:05.460504 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:05.460481    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp"
Jul 17 09:57:05.465419 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:05.465395    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp"]
Jul 17 09:57:05.569580 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:05.569518    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0-maas-api-service-ca\") pod \"50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0\" (UID: \"50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0\") "
Jul 17 09:57:05.569694 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:05.569682    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df8098ac-af38-4f40-ae90-b16b44ab1977-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp\" (UID: \"df8098ac-af38-4f40-ae90-b16b44ab1977\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp"
Jul 17 09:57:05.569877 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:05.569855    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0" (UID: "50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:57:05.670575 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:05.670542    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df8098ac-af38-4f40-ae90-b16b44ab1977-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp\" (UID: \"df8098ac-af38-4f40-ae90-b16b44ab1977\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp"
Jul 17 09:57:05.670677 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:05.670646    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:57:05.671089 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:05.671068    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df8098ac-af38-4f40-ae90-b16b44ab1977-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp\" (UID: \"df8098ac-af38-4f40-ae90-b16b44ab1977\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp"
Jul 17 09:57:05.773467 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:05.773444    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp"
Jul 17 09:57:06.111298 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:06.111272    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp"]
Jul 17 09:57:06.113622 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:57:06.113591    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf8098ac_af38_4f40_ae90_b16b44ab1977.slice/crio-4a0f9290baade38933574ef94b47d4cb2caffd283862778c89063c81454e515a WatchSource:0}: Error finding container 4a0f9290baade38933574ef94b47d4cb2caffd283862778c89063c81454e515a: Status 404 returned error can't find the container with id 4a0f9290baade38933574ef94b47d4cb2caffd283862778c89063c81454e515a
Jul 17 09:57:06.275625 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:06.275602    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-gr82w"
Jul 17 09:57:06.275711 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:06.275604    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-gr82w" event={"ID":"50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0","Type":"ContainerDied","Data":"e22dfe4a7581447e791ab00a2de87d2e077e871475c1a0696542edca6a97ea12"}
Jul 17 09:57:06.275772 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:06.275730    2574 scope.go:117] "RemoveContainer" containerID="ef7c31da0613e517aaf94cc85e49afa9b7237eb3ccf7891549a89126626089fa"
Jul 17 09:57:06.277076 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:06.276943    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp" event={"ID":"df8098ac-af38-4f40-ae90-b16b44ab1977","Type":"ContainerStarted","Data":"4a0f9290baade38933574ef94b47d4cb2caffd283862778c89063c81454e515a"}
Jul 17 09:57:06.298367 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:06.298345    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-gr82w"]
Jul 17 09:57:06.305784 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:06.305762    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-gr82w"]
Jul 17 09:57:06.897219 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:06.897189    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0" path="/var/lib/kubelet/pods/50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0/volumes"
Jul 17 09:57:07.284861 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:07.284827    2574 generic.go:356] "Generic (PLEG): container finished" podID="df8098ac-af38-4f40-ae90-b16b44ab1977" containerID="a07692e9c0d9ac9343d4b5f5053ab4edd658500478f221532679e16c445a1ce8" exitCode=7
Jul 17 09:57:07.285010 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:07.284912    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp" event={"ID":"df8098ac-af38-4f40-ae90-b16b44ab1977","Type":"ContainerDied","Data":"a07692e9c0d9ac9343d4b5f5053ab4edd658500478f221532679e16c445a1ce8"}
Jul 17 09:57:07.285061 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:07.285049    2574 scope.go:117] "RemoveContainer" containerID="a07692e9c0d9ac9343d4b5f5053ab4edd658500478f221532679e16c445a1ce8"
Jul 17 09:57:07.561740 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:07.561681    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh"]
Jul 17 09:57:07.562159 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:07.562146    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="50ace1ee-a9f1-4407-9e4d-36a6f7a69bd0" containerName="revoke-keys"
Jul 17 09:57:07.565165 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:07.565151    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh"
Jul 17 09:57:07.572225 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:07.572189    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh"]
Jul 17 09:57:07.687765 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:07.687733    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c67a6d3c-6e8a-4d6f-8596-3802465f7644-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh\" (UID: \"c67a6d3c-6e8a-4d6f-8596-3802465f7644\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh"
Jul 17 09:57:07.789175 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:07.789148    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c67a6d3c-6e8a-4d6f-8596-3802465f7644-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh\" (UID: \"c67a6d3c-6e8a-4d6f-8596-3802465f7644\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh"
Jul 17 09:57:07.789716 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:07.789697    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c67a6d3c-6e8a-4d6f-8596-3802465f7644-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh\" (UID: \"c67a6d3c-6e8a-4d6f-8596-3802465f7644\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh"
Jul 17 09:57:07.876626 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:07.876560    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh"
Jul 17 09:57:08.011220 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:08.011192    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh"]
Jul 17 09:57:08.013348 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:57:08.013317    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc67a6d3c_6e8a_4d6f_8596_3802465f7644.slice/crio-0343d9cb7e4db8367298ed8de5a346071b1cb8136557d24efe61b55377ab8480 WatchSource:0}: Error finding container 0343d9cb7e4db8367298ed8de5a346071b1cb8136557d24efe61b55377ab8480: Status 404 returned error can't find the container with id 0343d9cb7e4db8367298ed8de5a346071b1cb8136557d24efe61b55377ab8480
Jul 17 09:57:08.291656 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:08.291623    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh" event={"ID":"c67a6d3c-6e8a-4d6f-8596-3802465f7644","Type":"ContainerStarted","Data":"ce9cf4ecc810f5af628dbb622a5bc19bf2fa6a83e08f4ff700c05ac02b352946"}
Jul 17 09:57:08.291926 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:08.291683    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh" event={"ID":"c67a6d3c-6e8a-4d6f-8596-3802465f7644","Type":"ContainerStarted","Data":"0343d9cb7e4db8367298ed8de5a346071b1cb8136557d24efe61b55377ab8480"}
Jul 17 09:57:08.293477 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:08.293455    2574 generic.go:356] "Generic (PLEG): container finished" podID="df8098ac-af38-4f40-ae90-b16b44ab1977" containerID="eb927da2e773476543c37d04ff2ab934b61384d8d82f6f9ae681e575175dd7ef" exitCode=7
Jul 17 09:57:08.293551 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:08.293485    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp" event={"ID":"df8098ac-af38-4f40-ae90-b16b44ab1977","Type":"ContainerDied","Data":"eb927da2e773476543c37d04ff2ab934b61384d8d82f6f9ae681e575175dd7ef"}
Jul 17 09:57:08.293551 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:08.293537    2574 scope.go:117] "RemoveContainer" containerID="a07692e9c0d9ac9343d4b5f5053ab4edd658500478f221532679e16c445a1ce8"
Jul 17 09:57:08.293697 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:08.293683    2574 scope.go:117] "RemoveContainer" containerID="eb927da2e773476543c37d04ff2ab934b61384d8d82f6f9ae681e575175dd7ef"
Jul 17 09:57:08.293881 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:57:08.293859    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp_odh-ai-gateway-infra(df8098ac-af38-4f40-ae90-b16b44ab1977)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp" podUID="df8098ac-af38-4f40-ae90-b16b44ab1977"
Jul 17 09:57:08.306261 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:08.306198    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh" podStartSLOduration=1.306181767 podStartE2EDuration="1.306181767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:57:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:57:08.305326065 +0000 UTC m=+1825.887964458" watchObservedRunningTime="2026-07-17 09:57:08.306181767 +0000 UTC m=+1825.888820143"
Jul 17 09:57:09.303420 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:09.303399    2574 scope.go:117] "RemoveContainer" containerID="eb927da2e773476543c37d04ff2ab934b61384d8d82f6f9ae681e575175dd7ef"
Jul 17 09:57:09.303840 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:57:09.303566    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp_odh-ai-gateway-infra(df8098ac-af38-4f40-ae90-b16b44ab1977)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp" podUID="df8098ac-af38-4f40-ae90-b16b44ab1977"
Jul 17 09:57:09.305083 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:09.305054    2574 generic.go:356] "Generic (PLEG): container finished" podID="c67a6d3c-6e8a-4d6f-8596-3802465f7644" containerID="ce9cf4ecc810f5af628dbb622a5bc19bf2fa6a83e08f4ff700c05ac02b352946" exitCode=6
Jul 17 09:57:09.305184 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:09.305093    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh" event={"ID":"c67a6d3c-6e8a-4d6f-8596-3802465f7644","Type":"ContainerDied","Data":"ce9cf4ecc810f5af628dbb622a5bc19bf2fa6a83e08f4ff700c05ac02b352946"}
Jul 17 09:57:09.305292 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:09.305278    2574 scope.go:117] "RemoveContainer" containerID="ce9cf4ecc810f5af628dbb622a5bc19bf2fa6a83e08f4ff700c05ac02b352946"
Jul 17 09:57:10.311896 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:10.311864    2574 generic.go:356] "Generic (PLEG): container finished" podID="c67a6d3c-6e8a-4d6f-8596-3802465f7644" containerID="fe873ee7a6f4efd98f56bf501994e79f68bad081deb386cafba05d9c3241f9ad" exitCode=6
Jul 17 09:57:10.312305 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:10.311932    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh" event={"ID":"c67a6d3c-6e8a-4d6f-8596-3802465f7644","Type":"ContainerDied","Data":"fe873ee7a6f4efd98f56bf501994e79f68bad081deb386cafba05d9c3241f9ad"}
Jul 17 09:57:10.312305 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:10.311996    2574 scope.go:117] "RemoveContainer" containerID="ce9cf4ecc810f5af628dbb622a5bc19bf2fa6a83e08f4ff700c05ac02b352946"
Jul 17 09:57:10.312305 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:10.312138    2574 scope.go:117] "RemoveContainer" containerID="fe873ee7a6f4efd98f56bf501994e79f68bad081deb386cafba05d9c3241f9ad"
Jul 17 09:57:10.312498 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:57:10.312390    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh_odh-ai-gateway-infra(c67a6d3c-6e8a-4d6f-8596-3802465f7644)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh" podUID="c67a6d3c-6e8a-4d6f-8596-3802465f7644"
Jul 17 09:57:11.320518 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:11.320492    2574 scope.go:117] "RemoveContainer" containerID="fe873ee7a6f4efd98f56bf501994e79f68bad081deb386cafba05d9c3241f9ad"
Jul 17 09:57:11.320896 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:57:11.320692    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh_odh-ai-gateway-infra(c67a6d3c-6e8a-4d6f-8596-3802465f7644)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh" podUID="c67a6d3c-6e8a-4d6f-8596-3802465f7644"
Jul 17 09:57:22.895344 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:22.895311    2574 scope.go:117] "RemoveContainer" containerID="eb927da2e773476543c37d04ff2ab934b61384d8d82f6f9ae681e575175dd7ef"
Jul 17 09:57:23.386733 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:23.386698    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp" event={"ID":"df8098ac-af38-4f40-ae90-b16b44ab1977","Type":"ContainerStarted","Data":"9654c5242c5ad683bd56425c9e23f62da71ae34282d1e54714169415b3049c37"}
Jul 17 09:57:23.404741 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:23.404683    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp" podStartSLOduration=18.40466728 podStartE2EDuration="18.40466728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:57:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:57:23.402625783 +0000 UTC m=+1840.985264160" watchObservedRunningTime="2026-07-17 09:57:23.40466728 +0000 UTC m=+1840.987305708"
Jul 17 09:57:24.394528 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:24.394492    2574 generic.go:356] "Generic (PLEG): container finished" podID="df8098ac-af38-4f40-ae90-b16b44ab1977" containerID="9654c5242c5ad683bd56425c9e23f62da71ae34282d1e54714169415b3049c37" exitCode=7
Jul 17 09:57:24.394980 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:24.394563    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp" event={"ID":"df8098ac-af38-4f40-ae90-b16b44ab1977","Type":"ContainerDied","Data":"9654c5242c5ad683bd56425c9e23f62da71ae34282d1e54714169415b3049c37"}
Jul 17 09:57:24.394980 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:24.394624    2574 scope.go:117] "RemoveContainer" containerID="eb927da2e773476543c37d04ff2ab934b61384d8d82f6f9ae681e575175dd7ef"
Jul 17 09:57:24.394980 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:24.394929    2574 scope.go:117] "RemoveContainer" containerID="9654c5242c5ad683bd56425c9e23f62da71ae34282d1e54714169415b3049c37"
Jul 17 09:57:24.395269 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:57:24.395218    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp_odh-ai-gateway-infra(df8098ac-af38-4f40-ae90-b16b44ab1977)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp" podUID="df8098ac-af38-4f40-ae90-b16b44ab1977"
Jul 17 09:57:24.413317 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:24.413286    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp"]
Jul 17 09:57:24.892004 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:24.891968    2574 scope.go:117] "RemoveContainer" containerID="fe873ee7a6f4efd98f56bf501994e79f68bad081deb386cafba05d9c3241f9ad"
Jul 17 09:57:25.402668 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:25.402628    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh" event={"ID":"c67a6d3c-6e8a-4d6f-8596-3802465f7644","Type":"ContainerStarted","Data":"573ddd9c8f638a6d3b06d2cdaa6db13694f6ef9996ef736c76da5b842c163ba7"}
Jul 17 09:57:25.545625 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:25.545602    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp"
Jul 17 09:57:25.644257 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:25.644213    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df8098ac-af38-4f40-ae90-b16b44ab1977-maas-api-service-ca\") pod \"df8098ac-af38-4f40-ae90-b16b44ab1977\" (UID: \"df8098ac-af38-4f40-ae90-b16b44ab1977\") "
Jul 17 09:57:25.644580 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:25.644557    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df8098ac-af38-4f40-ae90-b16b44ab1977-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "df8098ac-af38-4f40-ae90-b16b44ab1977" (UID: "df8098ac-af38-4f40-ae90-b16b44ab1977"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:57:25.745742 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:25.745703    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/df8098ac-af38-4f40-ae90-b16b44ab1977-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:57:26.410449 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:26.410421    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp"
Jul 17 09:57:26.410924 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:26.410455    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp" event={"ID":"df8098ac-af38-4f40-ae90-b16b44ab1977","Type":"ContainerDied","Data":"4a0f9290baade38933574ef94b47d4cb2caffd283862778c89063c81454e515a"}
Jul 17 09:57:26.410924 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:26.410528    2574 scope.go:117] "RemoveContainer" containerID="9654c5242c5ad683bd56425c9e23f62da71ae34282d1e54714169415b3049c37"
Jul 17 09:57:26.412316 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:26.412290    2574 generic.go:356] "Generic (PLEG): container finished" podID="c67a6d3c-6e8a-4d6f-8596-3802465f7644" containerID="573ddd9c8f638a6d3b06d2cdaa6db13694f6ef9996ef736c76da5b842c163ba7" exitCode=6
Jul 17 09:57:26.412412 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:26.412328    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh" event={"ID":"c67a6d3c-6e8a-4d6f-8596-3802465f7644","Type":"ContainerDied","Data":"573ddd9c8f638a6d3b06d2cdaa6db13694f6ef9996ef736c76da5b842c163ba7"}
Jul 17 09:57:26.412567 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:26.412555    2574 scope.go:117] "RemoveContainer" containerID="573ddd9c8f638a6d3b06d2cdaa6db13694f6ef9996ef736c76da5b842c163ba7"
Jul 17 09:57:26.412770 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:57:26.412754    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh_odh-ai-gateway-infra(c67a6d3c-6e8a-4d6f-8596-3802465f7644)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh" podUID="c67a6d3c-6e8a-4d6f-8596-3802465f7644"
Jul 17 09:57:26.419787 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:26.419769    2574 scope.go:117] "RemoveContainer" containerID="fe873ee7a6f4efd98f56bf501994e79f68bad081deb386cafba05d9c3241f9ad"
Jul 17 09:57:26.428595 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:26.428574    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh"]
Jul 17 09:57:26.451272 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:26.451230    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp"]
Jul 17 09:57:26.457134 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:26.457106    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-66pkp"]
Jul 17 09:57:26.897606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:26.897522    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df8098ac-af38-4f40-ae90-b16b44ab1977" path="/var/lib/kubelet/pods/df8098ac-af38-4f40-ae90-b16b44ab1977/volumes"
Jul 17 09:57:27.557826 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:27.557804    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh"
Jul 17 09:57:27.664802 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:27.664770    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c67a6d3c-6e8a-4d6f-8596-3802465f7644-maas-api-service-ca\") pod \"c67a6d3c-6e8a-4d6f-8596-3802465f7644\" (UID: \"c67a6d3c-6e8a-4d6f-8596-3802465f7644\") "
Jul 17 09:57:27.665125 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:27.665103    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c67a6d3c-6e8a-4d6f-8596-3802465f7644-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c67a6d3c-6e8a-4d6f-8596-3802465f7644" (UID: "c67a6d3c-6e8a-4d6f-8596-3802465f7644"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:57:27.766008 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:27.765917    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c67a6d3c-6e8a-4d6f-8596-3802465f7644-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:57:28.427014 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:28.426978    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh" event={"ID":"c67a6d3c-6e8a-4d6f-8596-3802465f7644","Type":"ContainerDied","Data":"0343d9cb7e4db8367298ed8de5a346071b1cb8136557d24efe61b55377ab8480"}
Jul 17 09:57:28.427014 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:28.426978    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh"
Jul 17 09:57:28.427279 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:28.427076    2574 scope.go:117] "RemoveContainer" containerID="573ddd9c8f638a6d3b06d2cdaa6db13694f6ef9996ef736c76da5b842c163ba7"
Jul 17 09:57:28.449608 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:28.449564    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh"]
Jul 17 09:57:28.455515 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:28.455487    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-j8wsh"]
Jul 17 09:57:28.898453 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:28.898376    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c67a6d3c-6e8a-4d6f-8596-3802465f7644" path="/var/lib/kubelet/pods/c67a6d3c-6e8a-4d6f-8596-3802465f7644/volumes"
Jul 17 09:57:36.404482 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:36.404448    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9h5kq"]
Jul 17 09:57:36.405003 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:36.404988    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df8098ac-af38-4f40-ae90-b16b44ab1977" containerName="revoke-keys"
Jul 17 09:57:36.405053 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:36.405006    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="df8098ac-af38-4f40-ae90-b16b44ab1977" containerName="revoke-keys"
Jul 17 09:57:36.405053 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:36.405017    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c67a6d3c-6e8a-4d6f-8596-3802465f7644" containerName="revoke-keys"
Jul 17 09:57:36.405053 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:36.405025    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c67a6d3c-6e8a-4d6f-8596-3802465f7644" containerName="revoke-keys"
Jul 17 09:57:36.405053 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:36.405035    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c67a6d3c-6e8a-4d6f-8596-3802465f7644" containerName="revoke-keys"
Jul 17 09:57:36.405053 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:36.405041    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c67a6d3c-6e8a-4d6f-8596-3802465f7644" containerName="revoke-keys"
Jul 17 09:57:36.405053 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:36.405051    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df8098ac-af38-4f40-ae90-b16b44ab1977" containerName="revoke-keys"
Jul 17 09:57:36.405251 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:36.405056    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="df8098ac-af38-4f40-ae90-b16b44ab1977" containerName="revoke-keys"
Jul 17 09:57:36.405251 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:36.405062    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c67a6d3c-6e8a-4d6f-8596-3802465f7644" containerName="revoke-keys"
Jul 17 09:57:36.405251 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:36.405066    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c67a6d3c-6e8a-4d6f-8596-3802465f7644" containerName="revoke-keys"
Jul 17 09:57:36.405251 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:36.405138    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="df8098ac-af38-4f40-ae90-b16b44ab1977" containerName="revoke-keys"
Jul 17 09:57:36.405251 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:36.405146    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c67a6d3c-6e8a-4d6f-8596-3802465f7644" containerName="revoke-keys"
Jul 17 09:57:36.405251 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:36.405152    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="df8098ac-af38-4f40-ae90-b16b44ab1977" containerName="revoke-keys"
Jul 17 09:57:36.405251 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:36.405158    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="df8098ac-af38-4f40-ae90-b16b44ab1977" containerName="revoke-keys"
Jul 17 09:57:36.405251 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:36.405164    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c67a6d3c-6e8a-4d6f-8596-3802465f7644" containerName="revoke-keys"
Jul 17 09:57:36.405251 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:36.405174    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c67a6d3c-6e8a-4d6f-8596-3802465f7644" containerName="revoke-keys"
Jul 17 09:57:36.409848 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:36.409832    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9h5kq"
Jul 17 09:57:36.413002 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:36.412976    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 09:57:36.415087 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:36.415053    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9h5kq"]
Jul 17 09:57:36.451535 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:36.451497    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/87233f56-02e6-4070-994d-f0ac2681dc6b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-9h5kq\" (UID: \"87233f56-02e6-4070-994d-f0ac2681dc6b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9h5kq"
Jul 17 09:57:36.552946 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:36.552913    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/87233f56-02e6-4070-994d-f0ac2681dc6b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-9h5kq\" (UID: \"87233f56-02e6-4070-994d-f0ac2681dc6b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9h5kq"
Jul 17 09:57:36.553623 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:36.553600    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/87233f56-02e6-4070-994d-f0ac2681dc6b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-9h5kq\" (UID: \"87233f56-02e6-4070-994d-f0ac2681dc6b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9h5kq"
Jul 17 09:57:36.722625 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:36.722578    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9h5kq"
Jul 17 09:57:36.861031 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:36.861003    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9h5kq"]
Jul 17 09:57:36.862435 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:57:36.862406    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87233f56_02e6_4070_994d_f0ac2681dc6b.slice/crio-b6966796e8cce879f86173bf1f1d5a2fcb686798964ce402ee1b7725da1c7dab WatchSource:0}: Error finding container b6966796e8cce879f86173bf1f1d5a2fcb686798964ce402ee1b7725da1c7dab: Status 404 returned error can't find the container with id b6966796e8cce879f86173bf1f1d5a2fcb686798964ce402ee1b7725da1c7dab
Jul 17 09:57:37.477333 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:37.477296    2574 generic.go:356] "Generic (PLEG): container finished" podID="87233f56-02e6-4070-994d-f0ac2681dc6b" containerID="a6c7283f8597dc6a2db7c2c505d5a6d701cdd80a51e008af49a10fa5b0cfa73f" exitCode=6
Jul 17 09:57:37.477719 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:37.477367    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9h5kq" event={"ID":"87233f56-02e6-4070-994d-f0ac2681dc6b","Type":"ContainerDied","Data":"a6c7283f8597dc6a2db7c2c505d5a6d701cdd80a51e008af49a10fa5b0cfa73f"}
Jul 17 09:57:37.477719 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:37.477416    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9h5kq" event={"ID":"87233f56-02e6-4070-994d-f0ac2681dc6b","Type":"ContainerStarted","Data":"b6966796e8cce879f86173bf1f1d5a2fcb686798964ce402ee1b7725da1c7dab"}
Jul 17 09:57:37.477719 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:37.477637    2574 scope.go:117] "RemoveContainer" containerID="a6c7283f8597dc6a2db7c2c505d5a6d701cdd80a51e008af49a10fa5b0cfa73f"
Jul 17 09:57:38.484559 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:38.484530    2574 generic.go:356] "Generic (PLEG): container finished" podID="87233f56-02e6-4070-994d-f0ac2681dc6b" containerID="549803199ac94ea9775f2c13250cd26b9a01cf182fd76c50156508ee71b04d04" exitCode=6
Jul 17 09:57:38.485019 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:38.484620    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9h5kq" event={"ID":"87233f56-02e6-4070-994d-f0ac2681dc6b","Type":"ContainerDied","Data":"549803199ac94ea9775f2c13250cd26b9a01cf182fd76c50156508ee71b04d04"}
Jul 17 09:57:38.485019 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:38.484690    2574 scope.go:117] "RemoveContainer" containerID="a6c7283f8597dc6a2db7c2c505d5a6d701cdd80a51e008af49a10fa5b0cfa73f"
Jul 17 09:57:38.485019 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:38.484888    2574 scope.go:117] "RemoveContainer" containerID="549803199ac94ea9775f2c13250cd26b9a01cf182fd76c50156508ee71b04d04"
Jul 17 09:57:38.485148 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:57:38.485129    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-9h5kq_odh-ai-gateway-infra(87233f56-02e6-4070-994d-f0ac2681dc6b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9h5kq" podUID="87233f56-02e6-4070-994d-f0ac2681dc6b"
Jul 17 09:57:39.491701 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:39.491670    2574 scope.go:117] "RemoveContainer" containerID="549803199ac94ea9775f2c13250cd26b9a01cf182fd76c50156508ee71b04d04"
Jul 17 09:57:39.492162 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:57:39.491974    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-9h5kq_odh-ai-gateway-infra(87233f56-02e6-4070-994d-f0ac2681dc6b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9h5kq" podUID="87233f56-02e6-4070-994d-f0ac2681dc6b"
Jul 17 09:57:51.891486 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:51.891456    2574 scope.go:117] "RemoveContainer" containerID="549803199ac94ea9775f2c13250cd26b9a01cf182fd76c50156508ee71b04d04"
Jul 17 09:57:52.563940 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:52.563905    2574 generic.go:356] "Generic (PLEG): container finished" podID="87233f56-02e6-4070-994d-f0ac2681dc6b" containerID="6536485f185c9f69a1d5bc65c3a56579f3879dd2c9a5dc0e8f4dd00b6a3bf96f" exitCode=6
Jul 17 09:57:52.564095 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:52.563979    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9h5kq" event={"ID":"87233f56-02e6-4070-994d-f0ac2681dc6b","Type":"ContainerDied","Data":"6536485f185c9f69a1d5bc65c3a56579f3879dd2c9a5dc0e8f4dd00b6a3bf96f"}
Jul 17 09:57:52.564095 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:52.564040    2574 scope.go:117] "RemoveContainer" containerID="549803199ac94ea9775f2c13250cd26b9a01cf182fd76c50156508ee71b04d04"
Jul 17 09:57:52.564401 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:52.564378    2574 scope.go:117] "RemoveContainer" containerID="6536485f185c9f69a1d5bc65c3a56579f3879dd2c9a5dc0e8f4dd00b6a3bf96f"
Jul 17 09:57:52.564644 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:57:52.564626    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-9h5kq_odh-ai-gateway-infra(87233f56-02e6-4070-994d-f0ac2681dc6b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9h5kq" podUID="87233f56-02e6-4070-994d-f0ac2681dc6b"
Jul 17 09:57:53.590693 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:53.590660    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9h5kq"]
Jul 17 09:57:53.750731 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:53.750706    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9h5kq"
Jul 17 09:57:53.927023 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:53.926994    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/87233f56-02e6-4070-994d-f0ac2681dc6b-maas-api-service-ca\") pod \"87233f56-02e6-4070-994d-f0ac2681dc6b\" (UID: \"87233f56-02e6-4070-994d-f0ac2681dc6b\") "
Jul 17 09:57:53.927399 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:53.927373    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87233f56-02e6-4070-994d-f0ac2681dc6b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "87233f56-02e6-4070-994d-f0ac2681dc6b" (UID: "87233f56-02e6-4070-994d-f0ac2681dc6b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:57:54.028136 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:54.028097    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/87233f56-02e6-4070-994d-f0ac2681dc6b-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:57:54.583579 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:54.583545    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9h5kq"
Jul 17 09:57:54.583757 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:54.583584    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9h5kq" event={"ID":"87233f56-02e6-4070-994d-f0ac2681dc6b","Type":"ContainerDied","Data":"b6966796e8cce879f86173bf1f1d5a2fcb686798964ce402ee1b7725da1c7dab"}
Jul 17 09:57:54.583757 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:54.583655    2574 scope.go:117] "RemoveContainer" containerID="6536485f185c9f69a1d5bc65c3a56579f3879dd2c9a5dc0e8f4dd00b6a3bf96f"
Jul 17 09:57:54.607920 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:54.607878    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9h5kq"]
Jul 17 09:57:54.616925 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:54.616898    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9h5kq"]
Jul 17 09:57:54.897384 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:57:54.897306    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87233f56-02e6-4070-994d-f0ac2681dc6b" path="/var/lib/kubelet/pods/87233f56-02e6-4070-994d-f0ac2681dc6b/volumes"
Jul 17 09:58:05.502253 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:05.502148    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh"]
Jul 17 09:58:05.502850 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:05.502829    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87233f56-02e6-4070-994d-f0ac2681dc6b" containerName="revoke-keys"
Jul 17 09:58:05.502891 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:05.502855    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="87233f56-02e6-4070-994d-f0ac2681dc6b" containerName="revoke-keys"
Jul 17 09:58:05.502891 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:05.502877    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df8098ac-af38-4f40-ae90-b16b44ab1977" containerName="revoke-keys"
Jul 17 09:58:05.502891 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:05.502885    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="df8098ac-af38-4f40-ae90-b16b44ab1977" containerName="revoke-keys"
Jul 17 09:58:05.502979 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:05.502895    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87233f56-02e6-4070-994d-f0ac2681dc6b" containerName="revoke-keys"
Jul 17 09:58:05.502979 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:05.502904    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="87233f56-02e6-4070-994d-f0ac2681dc6b" containerName="revoke-keys"
Jul 17 09:58:05.503040 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:05.502997    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="87233f56-02e6-4070-994d-f0ac2681dc6b" containerName="revoke-keys"
Jul 17 09:58:05.503040 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:05.503009    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="87233f56-02e6-4070-994d-f0ac2681dc6b" containerName="revoke-keys"
Jul 17 09:58:05.503040 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:05.503017    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="87233f56-02e6-4070-994d-f0ac2681dc6b" containerName="revoke-keys"
Jul 17 09:58:05.508103 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:05.508070    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh"
Jul 17 09:58:05.510699 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:05.510668    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh"]
Jul 17 09:58:05.511231 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:05.511206    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 09:58:05.543449 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:05.543417    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d04b3158-0359-47dc-b795-2245cde060ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh\" (UID: \"d04b3158-0359-47dc-b795-2245cde060ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh"
Jul 17 09:58:05.644724 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:05.644690    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d04b3158-0359-47dc-b795-2245cde060ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh\" (UID: \"d04b3158-0359-47dc-b795-2245cde060ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh"
Jul 17 09:58:05.645335 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:05.645312    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d04b3158-0359-47dc-b795-2245cde060ac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh\" (UID: \"d04b3158-0359-47dc-b795-2245cde060ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh"
Jul 17 09:58:05.821162 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:05.821080    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh"
Jul 17 09:58:06.160821 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:06.160799    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh"]
Jul 17 09:58:06.162681 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:58:06.162652    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd04b3158_0359_47dc_b795_2245cde060ac.slice/crio-75b9f1172560d8a27f11b6d1f04b3c72aa214048ffb34d90a889e27ac03f0d88 WatchSource:0}: Error finding container 75b9f1172560d8a27f11b6d1f04b3c72aa214048ffb34d90a889e27ac03f0d88: Status 404 returned error can't find the container with id 75b9f1172560d8a27f11b6d1f04b3c72aa214048ffb34d90a889e27ac03f0d88
Jul 17 09:58:06.649841 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:06.649803    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh" event={"ID":"d04b3158-0359-47dc-b795-2245cde060ac","Type":"ContainerStarted","Data":"a0879b6de1ffc7a05f112781172c518ec8afbcb1d80578a910c28f1e7e116094"}
Jul 17 09:58:06.650215 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:06.649856    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh" event={"ID":"d04b3158-0359-47dc-b795-2245cde060ac","Type":"ContainerStarted","Data":"75b9f1172560d8a27f11b6d1f04b3c72aa214048ffb34d90a889e27ac03f0d88"}
Jul 17 09:58:06.664255 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:06.664146    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh" podStartSLOduration=1.664131248 podStartE2EDuration="1.664131248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:58:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:58:06.663613915 +0000 UTC m=+1884.246252290" watchObservedRunningTime="2026-07-17 09:58:06.664131248 +0000 UTC m=+1884.246769621"
Jul 17 09:58:07.607702 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:07.607648    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x7clb"]
Jul 17 09:58:07.608150 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:07.608136    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87233f56-02e6-4070-994d-f0ac2681dc6b" containerName="revoke-keys"
Jul 17 09:58:07.608195 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:07.608152    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="87233f56-02e6-4070-994d-f0ac2681dc6b" containerName="revoke-keys"
Jul 17 09:58:07.611379 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:07.611363    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x7clb"
Jul 17 09:58:07.617991 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:07.617964    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x7clb"]
Jul 17 09:58:07.657146 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:07.657115    2574 generic.go:356] "Generic (PLEG): container finished" podID="d04b3158-0359-47dc-b795-2245cde060ac" containerID="a0879b6de1ffc7a05f112781172c518ec8afbcb1d80578a910c28f1e7e116094" exitCode=7
Jul 17 09:58:07.657526 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:07.657172    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh" event={"ID":"d04b3158-0359-47dc-b795-2245cde060ac","Type":"ContainerDied","Data":"a0879b6de1ffc7a05f112781172c518ec8afbcb1d80578a910c28f1e7e116094"}
Jul 17 09:58:07.657526 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:07.657476    2574 scope.go:117] "RemoveContainer" containerID="a0879b6de1ffc7a05f112781172c518ec8afbcb1d80578a910c28f1e7e116094"
Jul 17 09:58:07.663937 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:07.663917    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b6584ec-437c-4578-9a10-1d2b078997f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-x7clb\" (UID: \"9b6584ec-437c-4578-9a10-1d2b078997f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x7clb"
Jul 17 09:58:07.764713 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:07.764684    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b6584ec-437c-4578-9a10-1d2b078997f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-x7clb\" (UID: \"9b6584ec-437c-4578-9a10-1d2b078997f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x7clb"
Jul 17 09:58:07.765309 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:07.765290    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b6584ec-437c-4578-9a10-1d2b078997f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-x7clb\" (UID: \"9b6584ec-437c-4578-9a10-1d2b078997f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x7clb"
Jul 17 09:58:07.923540 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:07.923520    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x7clb"
Jul 17 09:58:08.054422 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:08.054394    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x7clb"]
Jul 17 09:58:08.056610 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:58:08.056580    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b6584ec_437c_4578_9a10_1d2b078997f4.slice/crio-a3206842794595cc812cee053c453b534adf20b1e633220ea49b99e8cd19a94e WatchSource:0}: Error finding container a3206842794595cc812cee053c453b534adf20b1e633220ea49b99e8cd19a94e: Status 404 returned error can't find the container with id a3206842794595cc812cee053c453b534adf20b1e633220ea49b99e8cd19a94e
Jul 17 09:58:08.664095 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:08.664055    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x7clb" event={"ID":"9b6584ec-437c-4578-9a10-1d2b078997f4","Type":"ContainerStarted","Data":"f362ac19e740fe67746bef4ef0e4840bac3c78340e837f7b83d412b804a216cb"}
Jul 17 09:58:08.664561 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:08.664118    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x7clb" event={"ID":"9b6584ec-437c-4578-9a10-1d2b078997f4","Type":"ContainerStarted","Data":"a3206842794595cc812cee053c453b534adf20b1e633220ea49b99e8cd19a94e"}
Jul 17 09:58:08.665967 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:08.665940    2574 generic.go:356] "Generic (PLEG): container finished" podID="d04b3158-0359-47dc-b795-2245cde060ac" containerID="bf4ef7621e3fd84bff28e69f5487d205f4fe8fc86c74e68d53194ebcc9e436c5" exitCode=7
Jul 17 09:58:08.666074 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:08.665986    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh" event={"ID":"d04b3158-0359-47dc-b795-2245cde060ac","Type":"ContainerDied","Data":"bf4ef7621e3fd84bff28e69f5487d205f4fe8fc86c74e68d53194ebcc9e436c5"}
Jul 17 09:58:08.666074 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:08.666021    2574 scope.go:117] "RemoveContainer" containerID="a0879b6de1ffc7a05f112781172c518ec8afbcb1d80578a910c28f1e7e116094"
Jul 17 09:58:08.666220 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:08.666203    2574 scope.go:117] "RemoveContainer" containerID="bf4ef7621e3fd84bff28e69f5487d205f4fe8fc86c74e68d53194ebcc9e436c5"
Jul 17 09:58:08.666445 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:58:08.666425    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh_odh-ai-gateway-infra(d04b3158-0359-47dc-b795-2245cde060ac)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh" podUID="d04b3158-0359-47dc-b795-2245cde060ac"
Jul 17 09:58:08.679138 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:08.679098    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x7clb" podStartSLOduration=1.679082954 podStartE2EDuration="1.679082954s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:58:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:58:08.677764947 +0000 UTC m=+1886.260403322" watchObservedRunningTime="2026-07-17 09:58:08.679082954 +0000 UTC m=+1886.261721327"
Jul 17 09:58:09.673209 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:09.673167    2574 generic.go:356] "Generic (PLEG): container finished" podID="9b6584ec-437c-4578-9a10-1d2b078997f4" containerID="f362ac19e740fe67746bef4ef0e4840bac3c78340e837f7b83d412b804a216cb" exitCode=6
Jul 17 09:58:09.673660 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:09.673275    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x7clb" event={"ID":"9b6584ec-437c-4578-9a10-1d2b078997f4","Type":"ContainerDied","Data":"f362ac19e740fe67746bef4ef0e4840bac3c78340e837f7b83d412b804a216cb"}
Jul 17 09:58:09.673660 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:09.673515    2574 scope.go:117] "RemoveContainer" containerID="f362ac19e740fe67746bef4ef0e4840bac3c78340e837f7b83d412b804a216cb"
Jul 17 09:58:09.675104 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:09.675087    2574 scope.go:117] "RemoveContainer" containerID="bf4ef7621e3fd84bff28e69f5487d205f4fe8fc86c74e68d53194ebcc9e436c5"
Jul 17 09:58:09.675298 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:58:09.675274    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh_odh-ai-gateway-infra(d04b3158-0359-47dc-b795-2245cde060ac)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh" podUID="d04b3158-0359-47dc-b795-2245cde060ac"
Jul 17 09:58:10.682178 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:10.682142    2574 generic.go:356] "Generic (PLEG): container finished" podID="9b6584ec-437c-4578-9a10-1d2b078997f4" containerID="049f50a7cb1d407d3b2dbe4adad7b71e683f7f909ce8b265081533f29502af37" exitCode=6
Jul 17 09:58:10.682605 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:10.682189    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x7clb" event={"ID":"9b6584ec-437c-4578-9a10-1d2b078997f4","Type":"ContainerDied","Data":"049f50a7cb1d407d3b2dbe4adad7b71e683f7f909ce8b265081533f29502af37"}
Jul 17 09:58:10.682605 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:10.682261    2574 scope.go:117] "RemoveContainer" containerID="f362ac19e740fe67746bef4ef0e4840bac3c78340e837f7b83d412b804a216cb"
Jul 17 09:58:10.682605 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:10.682491    2574 scope.go:117] "RemoveContainer" containerID="049f50a7cb1d407d3b2dbe4adad7b71e683f7f909ce8b265081533f29502af37"
Jul 17 09:58:10.682761 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:58:10.682731    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-x7clb_odh-ai-gateway-infra(9b6584ec-437c-4578-9a10-1d2b078997f4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x7clb" podUID="9b6584ec-437c-4578-9a10-1d2b078997f4"
Jul 17 09:58:11.689553 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:11.689528    2574 scope.go:117] "RemoveContainer" containerID="049f50a7cb1d407d3b2dbe4adad7b71e683f7f909ce8b265081533f29502af37"
Jul 17 09:58:11.689928 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:58:11.689733    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-x7clb_odh-ai-gateway-infra(9b6584ec-437c-4578-9a10-1d2b078997f4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x7clb" podUID="9b6584ec-437c-4578-9a10-1d2b078997f4"
Jul 17 09:58:22.895252 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:22.895199    2574 scope.go:117] "RemoveContainer" containerID="049f50a7cb1d407d3b2dbe4adad7b71e683f7f909ce8b265081533f29502af37"
Jul 17 09:58:23.753738 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:23.753703    2574 generic.go:356] "Generic (PLEG): container finished" podID="9b6584ec-437c-4578-9a10-1d2b078997f4" containerID="f0c115a905f79e26b8267d387f6f5da4e64d2c0d833b1e3ce5f7095ca188016c" exitCode=6
Jul 17 09:58:23.753915 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:23.753780    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x7clb" event={"ID":"9b6584ec-437c-4578-9a10-1d2b078997f4","Type":"ContainerDied","Data":"f0c115a905f79e26b8267d387f6f5da4e64d2c0d833b1e3ce5f7095ca188016c"}
Jul 17 09:58:23.753915 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:23.753850    2574 scope.go:117] "RemoveContainer" containerID="049f50a7cb1d407d3b2dbe4adad7b71e683f7f909ce8b265081533f29502af37"
Jul 17 09:58:23.754210 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:23.754188    2574 scope.go:117] "RemoveContainer" containerID="f0c115a905f79e26b8267d387f6f5da4e64d2c0d833b1e3ce5f7095ca188016c"
Jul 17 09:58:23.754463 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:58:23.754441    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-x7clb_odh-ai-gateway-infra(9b6584ec-437c-4578-9a10-1d2b078997f4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x7clb" podUID="9b6584ec-437c-4578-9a10-1d2b078997f4"
Jul 17 09:58:23.891962 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:23.891931    2574 scope.go:117] "RemoveContainer" containerID="bf4ef7621e3fd84bff28e69f5487d205f4fe8fc86c74e68d53194ebcc9e436c5"
Jul 17 09:58:24.761860 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:24.761824    2574 generic.go:356] "Generic (PLEG): container finished" podID="d04b3158-0359-47dc-b795-2245cde060ac" containerID="70f7c82ed95f517b7e2c9345bd7c47f2e8aaf25db3d03d0941c9aa473d5b98a7" exitCode=7
Jul 17 09:58:24.762312 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:24.761898    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh" event={"ID":"d04b3158-0359-47dc-b795-2245cde060ac","Type":"ContainerDied","Data":"70f7c82ed95f517b7e2c9345bd7c47f2e8aaf25db3d03d0941c9aa473d5b98a7"}
Jul 17 09:58:24.762312 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:24.761957    2574 scope.go:117] "RemoveContainer" containerID="bf4ef7621e3fd84bff28e69f5487d205f4fe8fc86c74e68d53194ebcc9e436c5"
Jul 17 09:58:24.762312 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:24.762260    2574 scope.go:117] "RemoveContainer" containerID="70f7c82ed95f517b7e2c9345bd7c47f2e8aaf25db3d03d0941c9aa473d5b98a7"
Jul 17 09:58:24.762525 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:58:24.762502    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh_odh-ai-gateway-infra(d04b3158-0359-47dc-b795-2245cde060ac)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh" podUID="d04b3158-0359-47dc-b795-2245cde060ac"
Jul 17 09:58:24.784116 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:24.784092    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x7clb"]
Jul 17 09:58:24.923852 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:24.923827    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x7clb"
Jul 17 09:58:25.023493 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:25.023457    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b6584ec-437c-4578-9a10-1d2b078997f4-maas-api-service-ca\") pod \"9b6584ec-437c-4578-9a10-1d2b078997f4\" (UID: \"9b6584ec-437c-4578-9a10-1d2b078997f4\") "
Jul 17 09:58:25.023784 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:25.023761    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b6584ec-437c-4578-9a10-1d2b078997f4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9b6584ec-437c-4578-9a10-1d2b078997f4" (UID: "9b6584ec-437c-4578-9a10-1d2b078997f4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:58:25.124451 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:25.124420    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b6584ec-437c-4578-9a10-1d2b078997f4-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:58:25.768600 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:25.768571    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x7clb"
Jul 17 09:58:25.768600 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:25.768582    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x7clb" event={"ID":"9b6584ec-437c-4578-9a10-1d2b078997f4","Type":"ContainerDied","Data":"a3206842794595cc812cee053c453b534adf20b1e633220ea49b99e8cd19a94e"}
Jul 17 09:58:25.769106 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:25.768631    2574 scope.go:117] "RemoveContainer" containerID="f0c115a905f79e26b8267d387f6f5da4e64d2c0d833b1e3ce5f7095ca188016c"
Jul 17 09:58:25.787518 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:25.787492    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh"]
Jul 17 09:58:25.792256 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:25.792218    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x7clb"]
Jul 17 09:58:25.799870 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:25.799847    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x7clb"]
Jul 17 09:58:25.929695 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:25.929670    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh"
Jul 17 09:58:26.032353 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.032269    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d04b3158-0359-47dc-b795-2245cde060ac-maas-api-service-ca\") pod \"d04b3158-0359-47dc-b795-2245cde060ac\" (UID: \"d04b3158-0359-47dc-b795-2245cde060ac\") "
Jul 17 09:58:26.032563 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.032540    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d04b3158-0359-47dc-b795-2245cde060ac-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d04b3158-0359-47dc-b795-2245cde060ac" (UID: "d04b3158-0359-47dc-b795-2245cde060ac"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:58:26.133215 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.133173    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d04b3158-0359-47dc-b795-2245cde060ac-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:58:26.458706 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.458666    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9bsxk"]
Jul 17 09:58:26.459178 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.459160    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b6584ec-437c-4578-9a10-1d2b078997f4" containerName="revoke-keys"
Jul 17 09:58:26.459309 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.459181    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b6584ec-437c-4578-9a10-1d2b078997f4" containerName="revoke-keys"
Jul 17 09:58:26.459309 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.459192    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b6584ec-437c-4578-9a10-1d2b078997f4" containerName="revoke-keys"
Jul 17 09:58:26.459309 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.459201    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b6584ec-437c-4578-9a10-1d2b078997f4" containerName="revoke-keys"
Jul 17 09:58:26.459309 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.459209    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b6584ec-437c-4578-9a10-1d2b078997f4" containerName="revoke-keys"
Jul 17 09:58:26.459309 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.459216    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b6584ec-437c-4578-9a10-1d2b078997f4" containerName="revoke-keys"
Jul 17 09:58:26.459309 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.459262    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d04b3158-0359-47dc-b795-2245cde060ac" containerName="revoke-keys"
Jul 17 09:58:26.459309 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.459271    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d04b3158-0359-47dc-b795-2245cde060ac" containerName="revoke-keys"
Jul 17 09:58:26.459309 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.459283    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d04b3158-0359-47dc-b795-2245cde060ac" containerName="revoke-keys"
Jul 17 09:58:26.459309 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.459290    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d04b3158-0359-47dc-b795-2245cde060ac" containerName="revoke-keys"
Jul 17 09:58:26.459735 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.459415    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b6584ec-437c-4578-9a10-1d2b078997f4" containerName="revoke-keys"
Jul 17 09:58:26.459735 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.459429    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d04b3158-0359-47dc-b795-2245cde060ac" containerName="revoke-keys"
Jul 17 09:58:26.459735 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.459437    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b6584ec-437c-4578-9a10-1d2b078997f4" containerName="revoke-keys"
Jul 17 09:58:26.459735 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.459447    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b6584ec-437c-4578-9a10-1d2b078997f4" containerName="revoke-keys"
Jul 17 09:58:26.459735 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.459457    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d04b3158-0359-47dc-b795-2245cde060ac" containerName="revoke-keys"
Jul 17 09:58:26.462589 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.462570    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9bsxk"
Jul 17 09:58:26.470396 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.470370    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9bsxk"]
Jul 17 09:58:26.537468 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.537429    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f656c8e-1c6e-41cd-8783-944517a474ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-9bsxk\" (UID: \"8f656c8e-1c6e-41cd-8783-944517a474ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9bsxk"
Jul 17 09:58:26.638994 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.638961    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f656c8e-1c6e-41cd-8783-944517a474ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-9bsxk\" (UID: \"8f656c8e-1c6e-41cd-8783-944517a474ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9bsxk"
Jul 17 09:58:26.639610 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.639589    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f656c8e-1c6e-41cd-8783-944517a474ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-9bsxk\" (UID: \"8f656c8e-1c6e-41cd-8783-944517a474ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9bsxk"
Jul 17 09:58:26.774087 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.773986    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9bsxk"
Jul 17 09:58:26.778705 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.778682    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh"
Jul 17 09:58:26.778817 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.778714    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh" event={"ID":"d04b3158-0359-47dc-b795-2245cde060ac","Type":"ContainerDied","Data":"75b9f1172560d8a27f11b6d1f04b3c72aa214048ffb34d90a889e27ac03f0d88"}
Jul 17 09:58:26.778817 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.778797    2574 scope.go:117] "RemoveContainer" containerID="70f7c82ed95f517b7e2c9345bd7c47f2e8aaf25db3d03d0941c9aa473d5b98a7"
Jul 17 09:58:26.817278 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.817248    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh"]
Jul 17 09:58:26.821957 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.821929    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s9qbh"]
Jul 17 09:58:26.899970 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.899936    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b6584ec-437c-4578-9a10-1d2b078997f4" path="/var/lib/kubelet/pods/9b6584ec-437c-4578-9a10-1d2b078997f4/volumes"
Jul 17 09:58:26.900416 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.900395    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d04b3158-0359-47dc-b795-2245cde060ac" path="/var/lib/kubelet/pods/d04b3158-0359-47dc-b795-2245cde060ac/volumes"
Jul 17 09:58:26.915849 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:26.915829    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9bsxk"]
Jul 17 09:58:26.918701 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:58:26.918676    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f656c8e_1c6e_41cd_8783_944517a474ad.slice/crio-b27d5fd626689dee05580d028606be1a91ef5a7dbbd0b5e7121309a1a6c34d14 WatchSource:0}: Error finding container b27d5fd626689dee05580d028606be1a91ef5a7dbbd0b5e7121309a1a6c34d14: Status 404 returned error can't find the container with id b27d5fd626689dee05580d028606be1a91ef5a7dbbd0b5e7121309a1a6c34d14
Jul 17 09:58:27.785765 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:27.785726    2574 generic.go:356] "Generic (PLEG): container finished" podID="8f656c8e-1c6e-41cd-8783-944517a474ad" containerID="3eb859a9ec9f682ba1c652e23709f950251f9b2ed1775fcc38116df8b9d10b3e" exitCode=6
Jul 17 09:58:27.786225 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:27.785817    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9bsxk" event={"ID":"8f656c8e-1c6e-41cd-8783-944517a474ad","Type":"ContainerDied","Data":"3eb859a9ec9f682ba1c652e23709f950251f9b2ed1775fcc38116df8b9d10b3e"}
Jul 17 09:58:27.786225 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:27.785871    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9bsxk" event={"ID":"8f656c8e-1c6e-41cd-8783-944517a474ad","Type":"ContainerStarted","Data":"b27d5fd626689dee05580d028606be1a91ef5a7dbbd0b5e7121309a1a6c34d14"}
Jul 17 09:58:27.786225 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:27.785980    2574 scope.go:117] "RemoveContainer" containerID="3eb859a9ec9f682ba1c652e23709f950251f9b2ed1775fcc38116df8b9d10b3e"
Jul 17 09:58:28.794527 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:28.794492    2574 generic.go:356] "Generic (PLEG): container finished" podID="8f656c8e-1c6e-41cd-8783-944517a474ad" containerID="56ed0d068530cd9a8e9141a7e358e120708476c55e3a62bb93e418a29a3b4cdf" exitCode=6
Jul 17 09:58:28.794968 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:28.794574    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9bsxk" event={"ID":"8f656c8e-1c6e-41cd-8783-944517a474ad","Type":"ContainerDied","Data":"56ed0d068530cd9a8e9141a7e358e120708476c55e3a62bb93e418a29a3b4cdf"}
Jul 17 09:58:28.794968 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:28.794624    2574 scope.go:117] "RemoveContainer" containerID="3eb859a9ec9f682ba1c652e23709f950251f9b2ed1775fcc38116df8b9d10b3e"
Jul 17 09:58:28.794968 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:28.794850    2574 scope.go:117] "RemoveContainer" containerID="56ed0d068530cd9a8e9141a7e358e120708476c55e3a62bb93e418a29a3b4cdf"
Jul 17 09:58:28.795092 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:58:28.795063    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-9bsxk_odh-ai-gateway-infra(8f656c8e-1c6e-41cd-8783-944517a474ad)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9bsxk" podUID="8f656c8e-1c6e-41cd-8783-944517a474ad"
Jul 17 09:58:29.802101 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:29.802076    2574 scope.go:117] "RemoveContainer" containerID="56ed0d068530cd9a8e9141a7e358e120708476c55e3a62bb93e418a29a3b4cdf"
Jul 17 09:58:29.802507 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:58:29.802284    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-9bsxk_odh-ai-gateway-infra(8f656c8e-1c6e-41cd-8783-944517a474ad)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9bsxk" podUID="8f656c8e-1c6e-41cd-8783-944517a474ad"
Jul 17 09:58:43.892158 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:43.892129    2574 scope.go:117] "RemoveContainer" containerID="56ed0d068530cd9a8e9141a7e358e120708476c55e3a62bb93e418a29a3b4cdf"
Jul 17 09:58:44.882382 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:44.882292    2574 generic.go:356] "Generic (PLEG): container finished" podID="8f656c8e-1c6e-41cd-8783-944517a474ad" containerID="c995aa28dffd731c1ca5d7317832afe3a900b68a3fa526f3909d16fbe353a8ab" exitCode=6
Jul 17 09:58:44.882508 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:44.882374    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9bsxk" event={"ID":"8f656c8e-1c6e-41cd-8783-944517a474ad","Type":"ContainerDied","Data":"c995aa28dffd731c1ca5d7317832afe3a900b68a3fa526f3909d16fbe353a8ab"}
Jul 17 09:58:44.882508 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:44.882441    2574 scope.go:117] "RemoveContainer" containerID="56ed0d068530cd9a8e9141a7e358e120708476c55e3a62bb93e418a29a3b4cdf"
Jul 17 09:58:44.882671 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:44.882654    2574 scope.go:117] "RemoveContainer" containerID="c995aa28dffd731c1ca5d7317832afe3a900b68a3fa526f3909d16fbe353a8ab"
Jul 17 09:58:44.882893 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:58:44.882874    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-9bsxk_odh-ai-gateway-infra(8f656c8e-1c6e-41cd-8783-944517a474ad)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9bsxk" podUID="8f656c8e-1c6e-41cd-8783-944517a474ad"
Jul 17 09:58:45.908444 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:45.908411    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9bsxk"]
Jul 17 09:58:46.045465 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:46.045442    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9bsxk"
Jul 17 09:58:46.118379 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:46.118349    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f656c8e-1c6e-41cd-8783-944517a474ad-maas-api-service-ca\") pod \"8f656c8e-1c6e-41cd-8783-944517a474ad\" (UID: \"8f656c8e-1c6e-41cd-8783-944517a474ad\") "
Jul 17 09:58:46.118756 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:46.118734    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f656c8e-1c6e-41cd-8783-944517a474ad-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8f656c8e-1c6e-41cd-8783-944517a474ad" (UID: "8f656c8e-1c6e-41cd-8783-944517a474ad"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:58:46.219734 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:46.219653    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f656c8e-1c6e-41cd-8783-944517a474ad-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:58:46.895038 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:46.895006    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9bsxk"
Jul 17 09:58:46.897772 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:46.897745    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9bsxk" event={"ID":"8f656c8e-1c6e-41cd-8783-944517a474ad","Type":"ContainerDied","Data":"b27d5fd626689dee05580d028606be1a91ef5a7dbbd0b5e7121309a1a6c34d14"}
Jul 17 09:58:46.897941 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:46.897818    2574 scope.go:117] "RemoveContainer" containerID="c995aa28dffd731c1ca5d7317832afe3a900b68a3fa526f3909d16fbe353a8ab"
Jul 17 09:58:46.920893 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:46.920865    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9bsxk"]
Jul 17 09:58:46.926179 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:46.926151    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-9bsxk"]
Jul 17 09:58:48.897541 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:48.897506    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f656c8e-1c6e-41cd-8783-944517a474ad" path="/var/lib/kubelet/pods/8f656c8e-1c6e-41cd-8783-944517a474ad/volumes"
Jul 17 09:58:57.650424 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:57.650393    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x4c2k"]
Jul 17 09:58:57.650909 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:57.650802    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d04b3158-0359-47dc-b795-2245cde060ac" containerName="revoke-keys"
Jul 17 09:58:57.650909 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:57.650813    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d04b3158-0359-47dc-b795-2245cde060ac" containerName="revoke-keys"
Jul 17 09:58:57.650909 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:57.650823    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f656c8e-1c6e-41cd-8783-944517a474ad" containerName="revoke-keys"
Jul 17 09:58:57.650909 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:57.650829    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f656c8e-1c6e-41cd-8783-944517a474ad" containerName="revoke-keys"
Jul 17 09:58:57.650909 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:57.650839    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f656c8e-1c6e-41cd-8783-944517a474ad" containerName="revoke-keys"
Jul 17 09:58:57.650909 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:57.650843    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f656c8e-1c6e-41cd-8783-944517a474ad" containerName="revoke-keys"
Jul 17 09:58:57.650909 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:57.650913    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f656c8e-1c6e-41cd-8783-944517a474ad" containerName="revoke-keys"
Jul 17 09:58:57.651118 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:57.650919    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f656c8e-1c6e-41cd-8783-944517a474ad" containerName="revoke-keys"
Jul 17 09:58:57.651118 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:57.650929    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d04b3158-0359-47dc-b795-2245cde060ac" containerName="revoke-keys"
Jul 17 09:58:57.655600 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:57.655584    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x4c2k"
Jul 17 09:58:57.658490 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:57.658467    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 09:58:57.661390 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:57.661364    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x4c2k"]
Jul 17 09:58:57.732528 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:57.732498    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-x4c2k\" (UID: \"c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x4c2k"
Jul 17 09:58:57.833198 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:57.833162    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-x4c2k\" (UID: \"c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x4c2k"
Jul 17 09:58:57.833761 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:57.833741    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-x4c2k\" (UID: \"c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x4c2k"
Jul 17 09:58:57.967749 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:57.967718    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x4c2k"
Jul 17 09:58:58.102897 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:58.102859    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x4c2k"]
Jul 17 09:58:58.103420 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:58:58.103392    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2f2aa7c_9b2d_4921_8d82_8e281a6e2ff8.slice/crio-867333ea8cf1f93795beb0801364c72e202db64b721819c65bb97006ab8e1016 WatchSource:0}: Error finding container 867333ea8cf1f93795beb0801364c72e202db64b721819c65bb97006ab8e1016: Status 404 returned error can't find the container with id 867333ea8cf1f93795beb0801364c72e202db64b721819c65bb97006ab8e1016
Jul 17 09:58:58.961794 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:58.961752    2574 generic.go:356] "Generic (PLEG): container finished" podID="c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8" containerID="9638702887da2d3f5696b847681ec053c0ff682d31dade1114e283dc8b23b762" exitCode=6
Jul 17 09:58:58.962259 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:58.961810    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x4c2k" event={"ID":"c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8","Type":"ContainerDied","Data":"9638702887da2d3f5696b847681ec053c0ff682d31dade1114e283dc8b23b762"}
Jul 17 09:58:58.962259 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:58.961847    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x4c2k" event={"ID":"c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8","Type":"ContainerStarted","Data":"867333ea8cf1f93795beb0801364c72e202db64b721819c65bb97006ab8e1016"}
Jul 17 09:58:58.962259 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:58.962023    2574 scope.go:117] "RemoveContainer" containerID="9638702887da2d3f5696b847681ec053c0ff682d31dade1114e283dc8b23b762"
Jul 17 09:58:59.969439 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:59.969402    2574 generic.go:356] "Generic (PLEG): container finished" podID="c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8" containerID="46d2d310695560c94364f70f6a88b84e7b3a4ad09337126d9212d16fd65da71d" exitCode=6
Jul 17 09:58:59.969854 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:59.969489    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x4c2k" event={"ID":"c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8","Type":"ContainerDied","Data":"46d2d310695560c94364f70f6a88b84e7b3a4ad09337126d9212d16fd65da71d"}
Jul 17 09:58:59.969854 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:59.969551    2574 scope.go:117] "RemoveContainer" containerID="9638702887da2d3f5696b847681ec053c0ff682d31dade1114e283dc8b23b762"
Jul 17 09:58:59.969854 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:58:59.969773    2574 scope.go:117] "RemoveContainer" containerID="46d2d310695560c94364f70f6a88b84e7b3a4ad09337126d9212d16fd65da71d"
Jul 17 09:58:59.970072 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:58:59.970047    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-x4c2k_odh-ai-gateway-infra(c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x4c2k" podUID="c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8"
Jul 17 09:59:00.976329 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:00.976305    2574 scope.go:117] "RemoveContainer" containerID="46d2d310695560c94364f70f6a88b84e7b3a4ad09337126d9212d16fd65da71d"
Jul 17 09:59:00.976701 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:59:00.976502    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-x4c2k_odh-ai-gateway-infra(c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x4c2k" podUID="c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8"
Jul 17 09:59:05.545120 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:05.545087    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7m9fz"]
Jul 17 09:59:05.545751 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:05.545733    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f656c8e-1c6e-41cd-8783-944517a474ad" containerName="revoke-keys"
Jul 17 09:59:05.545811 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:05.545755    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f656c8e-1c6e-41cd-8783-944517a474ad" containerName="revoke-keys"
Jul 17 09:59:05.545916 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:05.545904    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f656c8e-1c6e-41cd-8783-944517a474ad" containerName="revoke-keys"
Jul 17 09:59:05.550659 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:05.550642    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7m9fz"
Jul 17 09:59:05.554929 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:05.554901    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7m9fz"]
Jul 17 09:59:05.707265 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:05.707207    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21eb5e1c-be86-4f87-814a-847df9e0d40f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-7m9fz\" (UID: \"21eb5e1c-be86-4f87-814a-847df9e0d40f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7m9fz"
Jul 17 09:59:05.808672 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:05.808583    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21eb5e1c-be86-4f87-814a-847df9e0d40f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-7m9fz\" (UID: \"21eb5e1c-be86-4f87-814a-847df9e0d40f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7m9fz"
Jul 17 09:59:05.809178 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:05.809158    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21eb5e1c-be86-4f87-814a-847df9e0d40f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-7m9fz\" (UID: \"21eb5e1c-be86-4f87-814a-847df9e0d40f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7m9fz"
Jul 17 09:59:05.864358 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:05.864324    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7m9fz"
Jul 17 09:59:06.007376 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:06.006297    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7m9fz"]
Jul 17 09:59:06.013514 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:59:06.013490    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21eb5e1c_be86_4f87_814a_847df9e0d40f.slice/crio-62cc0804d713a69fe70dfda5d78df251ee13659b0576ea1b8dad0ba97ba85f32 WatchSource:0}: Error finding container 62cc0804d713a69fe70dfda5d78df251ee13659b0576ea1b8dad0ba97ba85f32: Status 404 returned error can't find the container with id 62cc0804d713a69fe70dfda5d78df251ee13659b0576ea1b8dad0ba97ba85f32
Jul 17 09:59:07.009166 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:07.009128    2574 generic.go:356] "Generic (PLEG): container finished" podID="21eb5e1c-be86-4f87-814a-847df9e0d40f" containerID="880b17d38999b975b533fa2b20bbf7236c20271a51478848dbee1903788a97da" exitCode=7
Jul 17 09:59:07.009641 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:07.009217    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7m9fz" event={"ID":"21eb5e1c-be86-4f87-814a-847df9e0d40f","Type":"ContainerDied","Data":"880b17d38999b975b533fa2b20bbf7236c20271a51478848dbee1903788a97da"}
Jul 17 09:59:07.009641 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:07.009291    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7m9fz" event={"ID":"21eb5e1c-be86-4f87-814a-847df9e0d40f","Type":"ContainerStarted","Data":"62cc0804d713a69fe70dfda5d78df251ee13659b0576ea1b8dad0ba97ba85f32"}
Jul 17 09:59:07.009641 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:07.009392    2574 scope.go:117] "RemoveContainer" containerID="880b17d38999b975b533fa2b20bbf7236c20271a51478848dbee1903788a97da"
Jul 17 09:59:08.016364 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:08.016270    2574 generic.go:356] "Generic (PLEG): container finished" podID="21eb5e1c-be86-4f87-814a-847df9e0d40f" containerID="bd27a523880857368740ac003f906e4c0c976b9314768bea3149fe764196ca83" exitCode=7
Jul 17 09:59:08.016364 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:08.016331    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7m9fz" event={"ID":"21eb5e1c-be86-4f87-814a-847df9e0d40f","Type":"ContainerDied","Data":"bd27a523880857368740ac003f906e4c0c976b9314768bea3149fe764196ca83"}
Jul 17 09:59:08.016800 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:08.016393    2574 scope.go:117] "RemoveContainer" containerID="880b17d38999b975b533fa2b20bbf7236c20271a51478848dbee1903788a97da"
Jul 17 09:59:08.016800 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:08.016658    2574 scope.go:117] "RemoveContainer" containerID="bd27a523880857368740ac003f906e4c0c976b9314768bea3149fe764196ca83"
Jul 17 09:59:08.016916 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:59:08.016895    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-7m9fz_odh-ai-gateway-infra(21eb5e1c-be86-4f87-814a-847df9e0d40f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7m9fz" podUID="21eb5e1c-be86-4f87-814a-847df9e0d40f"
Jul 17 09:59:09.022802 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:09.022774    2574 scope.go:117] "RemoveContainer" containerID="bd27a523880857368740ac003f906e4c0c976b9314768bea3149fe764196ca83"
Jul 17 09:59:09.023193 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:59:09.022996    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-7m9fz_odh-ai-gateway-infra(21eb5e1c-be86-4f87-814a-847df9e0d40f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7m9fz" podUID="21eb5e1c-be86-4f87-814a-847df9e0d40f"
Jul 17 09:59:12.895707 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:12.895680    2574 scope.go:117] "RemoveContainer" containerID="46d2d310695560c94364f70f6a88b84e7b3a4ad09337126d9212d16fd65da71d"
Jul 17 09:59:14.049606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:14.049571    2574 generic.go:356] "Generic (PLEG): container finished" podID="c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8" containerID="c14f5168df4b014be4f8be0ccc2e94ece26afbec0f6ab502be94869a8bdcc891" exitCode=6
Jul 17 09:59:14.050041 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:14.049627    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x4c2k" event={"ID":"c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8","Type":"ContainerDied","Data":"c14f5168df4b014be4f8be0ccc2e94ece26afbec0f6ab502be94869a8bdcc891"}
Jul 17 09:59:14.050041 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:14.049674    2574 scope.go:117] "RemoveContainer" containerID="46d2d310695560c94364f70f6a88b84e7b3a4ad09337126d9212d16fd65da71d"
Jul 17 09:59:14.050041 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:14.049984    2574 scope.go:117] "RemoveContainer" containerID="c14f5168df4b014be4f8be0ccc2e94ece26afbec0f6ab502be94869a8bdcc891"
Jul 17 09:59:14.050279 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:59:14.050257    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-x4c2k_odh-ai-gateway-infra(c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x4c2k" podUID="c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8"
Jul 17 09:59:15.075916 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:15.075884    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x4c2k"]
Jul 17 09:59:15.211437 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:15.211416    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x4c2k"
Jul 17 09:59:15.304331 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:15.304302    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8-maas-api-service-ca\") pod \"c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8\" (UID: \"c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8\") "
Jul 17 09:59:15.304628 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:15.304606    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8" (UID: "c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:59:15.405281 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:15.405167    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:59:16.062732 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:16.062695    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x4c2k" event={"ID":"c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8","Type":"ContainerDied","Data":"867333ea8cf1f93795beb0801364c72e202db64b721819c65bb97006ab8e1016"}
Jul 17 09:59:16.062732 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:16.062725    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x4c2k"
Jul 17 09:59:16.062984 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:16.062755    2574 scope.go:117] "RemoveContainer" containerID="c14f5168df4b014be4f8be0ccc2e94ece26afbec0f6ab502be94869a8bdcc891"
Jul 17 09:59:16.089373 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:16.089343    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x4c2k"]
Jul 17 09:59:16.097771 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:16.097747    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-x4c2k"]
Jul 17 09:59:16.898302 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:16.898272    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8" path="/var/lib/kubelet/pods/c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8/volumes"
Jul 17 09:59:20.892200 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:20.892168    2574 scope.go:117] "RemoveContainer" containerID="bd27a523880857368740ac003f906e4c0c976b9314768bea3149fe764196ca83"
Jul 17 09:59:22.098162 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:22.098132    2574 generic.go:356] "Generic (PLEG): container finished" podID="21eb5e1c-be86-4f87-814a-847df9e0d40f" containerID="05a041ef9f5770329c36d8ddee5c097380b8be82b22aed4b72a51144f5de9e65" exitCode=7
Jul 17 09:59:22.098565 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:22.098205    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7m9fz" event={"ID":"21eb5e1c-be86-4f87-814a-847df9e0d40f","Type":"ContainerDied","Data":"05a041ef9f5770329c36d8ddee5c097380b8be82b22aed4b72a51144f5de9e65"}
Jul 17 09:59:22.098565 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:22.098279    2574 scope.go:117] "RemoveContainer" containerID="bd27a523880857368740ac003f906e4c0c976b9314768bea3149fe764196ca83"
Jul 17 09:59:22.098565 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:22.098518    2574 scope.go:117] "RemoveContainer" containerID="05a041ef9f5770329c36d8ddee5c097380b8be82b22aed4b72a51144f5de9e65"
Jul 17 09:59:22.098758 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:59:22.098734    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-7m9fz_odh-ai-gateway-infra(21eb5e1c-be86-4f87-814a-847df9e0d40f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7m9fz" podUID="21eb5e1c-be86-4f87-814a-847df9e0d40f"
Jul 17 09:59:23.128988 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:23.128957    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7m9fz"]
Jul 17 09:59:23.263960 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:23.263938    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7m9fz"
Jul 17 09:59:23.268493 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:23.268473    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21eb5e1c-be86-4f87-814a-847df9e0d40f-maas-api-service-ca\") pod \"21eb5e1c-be86-4f87-814a-847df9e0d40f\" (UID: \"21eb5e1c-be86-4f87-814a-847df9e0d40f\") "
Jul 17 09:59:23.268834 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:23.268811    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21eb5e1c-be86-4f87-814a-847df9e0d40f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "21eb5e1c-be86-4f87-814a-847df9e0d40f" (UID: "21eb5e1c-be86-4f87-814a-847df9e0d40f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:59:23.370069 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:23.370046    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21eb5e1c-be86-4f87-814a-847df9e0d40f-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:59:24.110452 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:24.110427    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7m9fz"
Jul 17 09:59:24.110598 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:24.110445    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7m9fz" event={"ID":"21eb5e1c-be86-4f87-814a-847df9e0d40f","Type":"ContainerDied","Data":"62cc0804d713a69fe70dfda5d78df251ee13659b0576ea1b8dad0ba97ba85f32"}
Jul 17 09:59:24.110598 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:24.110510    2574 scope.go:117] "RemoveContainer" containerID="05a041ef9f5770329c36d8ddee5c097380b8be82b22aed4b72a51144f5de9e65"
Jul 17 09:59:24.130604 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:24.130585    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7m9fz"]
Jul 17 09:59:24.132560 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:24.132540    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7m9fz"]
Jul 17 09:59:24.898467 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:24.898434    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21eb5e1c-be86-4f87-814a-847df9e0d40f" path="/var/lib/kubelet/pods/21eb5e1c-be86-4f87-814a-847df9e0d40f/volumes"
Jul 17 09:59:26.508912 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:26.508880    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc"]
Jul 17 09:59:26.509342 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:26.509327    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21eb5e1c-be86-4f87-814a-847df9e0d40f" containerName="revoke-keys"
Jul 17 09:59:26.509342 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:26.509340    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="21eb5e1c-be86-4f87-814a-847df9e0d40f" containerName="revoke-keys"
Jul 17 09:59:26.509430 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:26.509355    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8" containerName="revoke-keys"
Jul 17 09:59:26.509430 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:26.509360    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8" containerName="revoke-keys"
Jul 17 09:59:26.509430 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:26.509371    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8" containerName="revoke-keys"
Jul 17 09:59:26.509430 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:26.509376    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8" containerName="revoke-keys"
Jul 17 09:59:26.509430 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:26.509398    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21eb5e1c-be86-4f87-814a-847df9e0d40f" containerName="revoke-keys"
Jul 17 09:59:26.509430 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:26.509404    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="21eb5e1c-be86-4f87-814a-847df9e0d40f" containerName="revoke-keys"
Jul 17 09:59:26.509430 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:26.509410    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21eb5e1c-be86-4f87-814a-847df9e0d40f" containerName="revoke-keys"
Jul 17 09:59:26.509430 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:26.509417    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="21eb5e1c-be86-4f87-814a-847df9e0d40f" containerName="revoke-keys"
Jul 17 09:59:26.509666 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:26.509483    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="21eb5e1c-be86-4f87-814a-847df9e0d40f" containerName="revoke-keys"
Jul 17 09:59:26.509666 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:26.509491    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="21eb5e1c-be86-4f87-814a-847df9e0d40f" containerName="revoke-keys"
Jul 17 09:59:26.509666 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:26.509497    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8" containerName="revoke-keys"
Jul 17 09:59:26.509666 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:26.509503    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8" containerName="revoke-keys"
Jul 17 09:59:26.514156 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:26.514140    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc"
Jul 17 09:59:26.521861 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:26.521841    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 09:59:26.525342 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:26.525320    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc"]
Jul 17 09:59:26.596161 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:26.596139    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc\" (UID: \"a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc"
Jul 17 09:59:26.697038 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:26.697015    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc\" (UID: \"a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc"
Jul 17 09:59:26.697588 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:26.697571    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc\" (UID: \"a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc"
Jul 17 09:59:26.823962 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:26.823906    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc"
Jul 17 09:59:26.982507 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:26.982484    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc"]
Jul 17 09:59:26.985576 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:59:26.985553    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda48c36ae_2a00_47cf_9c8b_5ee0cf2be5e5.slice/crio-ff129a8f85274622c9e5b0a818d26b3b0fce652c34536483e5dc4ac39c63c77c WatchSource:0}: Error finding container ff129a8f85274622c9e5b0a818d26b3b0fce652c34536483e5dc4ac39c63c77c: Status 404 returned error can't find the container with id ff129a8f85274622c9e5b0a818d26b3b0fce652c34536483e5dc4ac39c63c77c
Jul 17 09:59:27.127954 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:27.127921    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc" event={"ID":"a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5","Type":"ContainerStarted","Data":"ff129a8f85274622c9e5b0a818d26b3b0fce652c34536483e5dc4ac39c63c77c"}
Jul 17 09:59:28.133601 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:28.133563    2574 generic.go:356] "Generic (PLEG): container finished" podID="a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5" containerID="adb0e8abcd77ce1250899e90a2f61d47135f6ff5ae76ad7094e47b41fd5a3e0d" exitCode=6
Jul 17 09:59:28.134009 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:28.133639    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc" event={"ID":"a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5","Type":"ContainerDied","Data":"adb0e8abcd77ce1250899e90a2f61d47135f6ff5ae76ad7094e47b41fd5a3e0d"}
Jul 17 09:59:28.134009 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:28.133805    2574 scope.go:117] "RemoveContainer" containerID="adb0e8abcd77ce1250899e90a2f61d47135f6ff5ae76ad7094e47b41fd5a3e0d"
Jul 17 09:59:29.140879 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:29.140801    2574 generic.go:356] "Generic (PLEG): container finished" podID="a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5" containerID="74a8f1fd477c5d47c2596191e0e8d143cff57bacc5e77e3cf9c7a0894c797ae9" exitCode=6
Jul 17 09:59:29.141301 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:29.140876    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc" event={"ID":"a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5","Type":"ContainerDied","Data":"74a8f1fd477c5d47c2596191e0e8d143cff57bacc5e77e3cf9c7a0894c797ae9"}
Jul 17 09:59:29.141301 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:29.140933    2574 scope.go:117] "RemoveContainer" containerID="adb0e8abcd77ce1250899e90a2f61d47135f6ff5ae76ad7094e47b41fd5a3e0d"
Jul 17 09:59:29.141301 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:29.141025    2574 scope.go:117] "RemoveContainer" containerID="74a8f1fd477c5d47c2596191e0e8d143cff57bacc5e77e3cf9c7a0894c797ae9"
Jul 17 09:59:29.141301 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:59:29.141205    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc_odh-ai-gateway-infra(a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc" podUID="a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5"
Jul 17 09:59:30.147913 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:30.147893    2574 scope.go:117] "RemoveContainer" containerID="74a8f1fd477c5d47c2596191e0e8d143cff57bacc5e77e3cf9c7a0894c797ae9"
Jul 17 09:59:30.148270 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:59:30.148050    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc_odh-ai-gateway-infra(a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc" podUID="a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5"
Jul 17 09:59:40.891904 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:40.891822    2574 scope.go:117] "RemoveContainer" containerID="74a8f1fd477c5d47c2596191e0e8d143cff57bacc5e77e3cf9c7a0894c797ae9"
Jul 17 09:59:41.207734 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:41.207706    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc" event={"ID":"a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5","Type":"ContainerStarted","Data":"1246379c527b953075c685de7e81e53dc12b4702df36607e6b2d3fe8050be2cd"}
Jul 17 09:59:41.232644 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:41.232600    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc" podStartSLOduration=15.232587459 podStartE2EDuration="15.232587459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:59:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 09:59:41.23126639 +0000 UTC m=+1978.813904762" watchObservedRunningTime="2026-07-17 09:59:41.232587459 +0000 UTC m=+1978.815225833"
Jul 17 09:59:42.214068 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:42.214035    2574 generic.go:356] "Generic (PLEG): container finished" podID="a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5" containerID="1246379c527b953075c685de7e81e53dc12b4702df36607e6b2d3fe8050be2cd" exitCode=6
Jul 17 09:59:42.214477 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:42.214114    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc" event={"ID":"a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5","Type":"ContainerDied","Data":"1246379c527b953075c685de7e81e53dc12b4702df36607e6b2d3fe8050be2cd"}
Jul 17 09:59:42.214477 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:42.214175    2574 scope.go:117] "RemoveContainer" containerID="74a8f1fd477c5d47c2596191e0e8d143cff57bacc5e77e3cf9c7a0894c797ae9"
Jul 17 09:59:42.214477 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:42.214387    2574 scope.go:117] "RemoveContainer" containerID="1246379c527b953075c685de7e81e53dc12b4702df36607e6b2d3fe8050be2cd"
Jul 17 09:59:42.214608 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:59:42.214561    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc_odh-ai-gateway-infra(a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc" podUID="a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5"
Jul 17 09:59:42.259088 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:42.259068    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc"]
Jul 17 09:59:43.362137 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:43.362116    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc"
Jul 17 09:59:43.428985 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:43.428922    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5-maas-api-service-ca\") pod \"a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5\" (UID: \"a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5\") "
Jul 17 09:59:43.429231 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:43.429211    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5" (UID: "a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 09:59:43.530071 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:43.530050    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 09:59:44.226322 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:44.226291    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc"
Jul 17 09:59:44.226322 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:44.226307    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc" event={"ID":"a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5","Type":"ContainerDied","Data":"ff129a8f85274622c9e5b0a818d26b3b0fce652c34536483e5dc4ac39c63c77c"}
Jul 17 09:59:44.226536 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:44.226370    2574 scope.go:117] "RemoveContainer" containerID="1246379c527b953075c685de7e81e53dc12b4702df36607e6b2d3fe8050be2cd"
Jul 17 09:59:44.261170 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:44.261146    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc"]
Jul 17 09:59:44.266191 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:44.266171    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-zlfvc"]
Jul 17 09:59:44.897790 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:44.897752    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5" path="/var/lib/kubelet/pods/a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5/volumes"
Jul 17 09:59:47.712145 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:47.712107    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-brk6f"]
Jul 17 09:59:47.712632 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:47.712587    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5" containerName="revoke-keys"
Jul 17 09:59:47.712632 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:47.712601    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5" containerName="revoke-keys"
Jul 17 09:59:47.712632 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:47.712626    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5" containerName="revoke-keys"
Jul 17 09:59:47.712632 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:47.712635    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5" containerName="revoke-keys"
Jul 17 09:59:47.712853 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:47.712644    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5" containerName="revoke-keys"
Jul 17 09:59:47.712853 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:47.712649    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5" containerName="revoke-keys"
Jul 17 09:59:47.712853 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:47.712656    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8" containerName="revoke-keys"
Jul 17 09:59:47.712853 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:47.712662    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8" containerName="revoke-keys"
Jul 17 09:59:47.712853 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:47.712738    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5" containerName="revoke-keys"
Jul 17 09:59:47.712853 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:47.712747    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5" containerName="revoke-keys"
Jul 17 09:59:47.712853 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:47.712754    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="21eb5e1c-be86-4f87-814a-847df9e0d40f" containerName="revoke-keys"
Jul 17 09:59:47.712853 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:47.712761    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2f2aa7c-9b2d-4921-8d82-8e281a6e2ff8" containerName="revoke-keys"
Jul 17 09:59:47.715965 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:47.715940    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-brk6f"
Jul 17 09:59:47.719553 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:47.719525    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 09:59:47.727738 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:47.727717    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-brk6f"]
Jul 17 09:59:47.764681 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:47.764648    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0d15ca8-cbc4-483d-a5f2-4eed7ad68746-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-brk6f\" (UID: \"e0d15ca8-cbc4-483d-a5f2-4eed7ad68746\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-brk6f"
Jul 17 09:59:47.865606 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:47.865575    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0d15ca8-cbc4-483d-a5f2-4eed7ad68746-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-brk6f\" (UID: \"e0d15ca8-cbc4-483d-a5f2-4eed7ad68746\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-brk6f"
Jul 17 09:59:47.866137 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:47.866120    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0d15ca8-cbc4-483d-a5f2-4eed7ad68746-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-brk6f\" (UID: \"e0d15ca8-cbc4-483d-a5f2-4eed7ad68746\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-brk6f"
Jul 17 09:59:48.026038 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:48.025988    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-brk6f"
Jul 17 09:59:48.164440 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:48.164416    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-brk6f"]
Jul 17 09:59:48.166816 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:59:48.166789    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0d15ca8_cbc4_483d_a5f2_4eed7ad68746.slice/crio-60ae973a870e7640aac8f9ebc4f0daccf620b4e534924c15f1190fa818215eaa WatchSource:0}: Error finding container 60ae973a870e7640aac8f9ebc4f0daccf620b4e534924c15f1190fa818215eaa: Status 404 returned error can't find the container with id 60ae973a870e7640aac8f9ebc4f0daccf620b4e534924c15f1190fa818215eaa
Jul 17 09:59:48.249144 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:48.249114    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-brk6f" event={"ID":"e0d15ca8-cbc4-483d-a5f2-4eed7ad68746","Type":"ContainerStarted","Data":"60ae973a870e7640aac8f9ebc4f0daccf620b4e534924c15f1190fa818215eaa"}
Jul 17 09:59:49.262895 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:49.262864    2574 generic.go:356] "Generic (PLEG): container finished" podID="e0d15ca8-cbc4-483d-a5f2-4eed7ad68746" containerID="a50335163c5214d13b46467933bff8268632e61d9d5de7dc457d5d943a3618d2" exitCode=6
Jul 17 09:59:49.263255 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:49.262938    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-brk6f" event={"ID":"e0d15ca8-cbc4-483d-a5f2-4eed7ad68746","Type":"ContainerDied","Data":"a50335163c5214d13b46467933bff8268632e61d9d5de7dc457d5d943a3618d2"}
Jul 17 09:59:49.263255 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:49.263064    2574 scope.go:117] "RemoveContainer" containerID="a50335163c5214d13b46467933bff8268632e61d9d5de7dc457d5d943a3618d2"
Jul 17 09:59:50.269578 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:50.269549    2574 generic.go:356] "Generic (PLEG): container finished" podID="e0d15ca8-cbc4-483d-a5f2-4eed7ad68746" containerID="04c78df9b02c16cacbdda29f8b9b44415ce7859d94711caa6824de1886f14c20" exitCode=6
Jul 17 09:59:50.269924 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:50.269628    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-brk6f" event={"ID":"e0d15ca8-cbc4-483d-a5f2-4eed7ad68746","Type":"ContainerDied","Data":"04c78df9b02c16cacbdda29f8b9b44415ce7859d94711caa6824de1886f14c20"}
Jul 17 09:59:50.269924 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:50.269681    2574 scope.go:117] "RemoveContainer" containerID="a50335163c5214d13b46467933bff8268632e61d9d5de7dc457d5d943a3618d2"
Jul 17 09:59:50.269924 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:50.269878    2574 scope.go:117] "RemoveContainer" containerID="04c78df9b02c16cacbdda29f8b9b44415ce7859d94711caa6824de1886f14c20"
Jul 17 09:59:50.270069 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:59:50.270051    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-brk6f_odh-ai-gateway-infra(e0d15ca8-cbc4-483d-a5f2-4eed7ad68746)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-brk6f" podUID="e0d15ca8-cbc4-483d-a5f2-4eed7ad68746"
Jul 17 09:59:51.275879 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:51.275852    2574 scope.go:117] "RemoveContainer" containerID="04c78df9b02c16cacbdda29f8b9b44415ce7859d94711caa6824de1886f14c20"
Jul 17 09:59:51.276216 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:59:51.276038    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-brk6f_odh-ai-gateway-infra(e0d15ca8-cbc4-483d-a5f2-4eed7ad68746)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-brk6f" podUID="e0d15ca8-cbc4-483d-a5f2-4eed7ad68746"
Jul 17 09:59:55.605989 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:55.605955    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4"]
Jul 17 09:59:55.606673 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:55.606652    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="a48c36ae-2a00-47cf-9c8b-5ee0cf2be5e5" containerName="revoke-keys"
Jul 17 09:59:55.611133 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:55.611113    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4"
Jul 17 09:59:55.625756 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:55.625735    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4"]
Jul 17 09:59:55.726971 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:55.726948    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a35a09e-a71c-4768-a6f9-500e8251d517-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4\" (UID: \"5a35a09e-a71c-4768-a6f9-500e8251d517\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4"
Jul 17 09:59:55.827298 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:55.827269    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a35a09e-a71c-4768-a6f9-500e8251d517-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4\" (UID: \"5a35a09e-a71c-4768-a6f9-500e8251d517\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4"
Jul 17 09:59:55.827843 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:55.827823    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a35a09e-a71c-4768-a6f9-500e8251d517-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4\" (UID: \"5a35a09e-a71c-4768-a6f9-500e8251d517\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4"
Jul 17 09:59:55.921273 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:55.921231    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4"
Jul 17 09:59:56.063696 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:56.063673    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4"]
Jul 17 09:59:56.065749 ip-10-0-136-235 kubenswrapper[2574]: W0717 09:59:56.065723    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a35a09e_a71c_4768_a6f9_500e8251d517.slice/crio-685b086331ffeadc811f090ab97e45e6e8e0a2bb69165233efe5d61ad54a5e47 WatchSource:0}: Error finding container 685b086331ffeadc811f090ab97e45e6e8e0a2bb69165233efe5d61ad54a5e47: Status 404 returned error can't find the container with id 685b086331ffeadc811f090ab97e45e6e8e0a2bb69165233efe5d61ad54a5e47
Jul 17 09:59:56.302234 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:56.302170    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4" event={"ID":"5a35a09e-a71c-4768-a6f9-500e8251d517","Type":"ContainerStarted","Data":"685b086331ffeadc811f090ab97e45e6e8e0a2bb69165233efe5d61ad54a5e47"}
Jul 17 09:59:57.308622 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:57.308587    2574 generic.go:356] "Generic (PLEG): container finished" podID="5a35a09e-a71c-4768-a6f9-500e8251d517" containerID="b8916b86bf13379cc2e561ba158d63c5ea34f9601beb7fb16acad405a1890a90" exitCode=7
Jul 17 09:59:57.309043 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:57.308661    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4" event={"ID":"5a35a09e-a71c-4768-a6f9-500e8251d517","Type":"ContainerDied","Data":"b8916b86bf13379cc2e561ba158d63c5ea34f9601beb7fb16acad405a1890a90"}
Jul 17 09:59:57.309043 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:57.308853    2574 scope.go:117] "RemoveContainer" containerID="b8916b86bf13379cc2e561ba158d63c5ea34f9601beb7fb16acad405a1890a90"
Jul 17 09:59:58.315953 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:58.315922    2574 generic.go:356] "Generic (PLEG): container finished" podID="5a35a09e-a71c-4768-a6f9-500e8251d517" containerID="00b400539437b2ac80ed5d81036d7d759f22e7a281b9516d9efd1a852aeb47ab" exitCode=7
Jul 17 09:59:58.316420 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:58.315996    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4" event={"ID":"5a35a09e-a71c-4768-a6f9-500e8251d517","Type":"ContainerDied","Data":"00b400539437b2ac80ed5d81036d7d759f22e7a281b9516d9efd1a852aeb47ab"}
Jul 17 09:59:58.316420 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:58.316053    2574 scope.go:117] "RemoveContainer" containerID="b8916b86bf13379cc2e561ba158d63c5ea34f9601beb7fb16acad405a1890a90"
Jul 17 09:59:58.316420 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:58.316205    2574 scope.go:117] "RemoveContainer" containerID="00b400539437b2ac80ed5d81036d7d759f22e7a281b9516d9efd1a852aeb47ab"
Jul 17 09:59:58.316582 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:59:58.316454    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4_odh-ai-gateway-infra(5a35a09e-a71c-4768-a6f9-500e8251d517)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4" podUID="5a35a09e-a71c-4768-a6f9-500e8251d517"
Jul 17 09:59:59.322996 ip-10-0-136-235 kubenswrapper[2574]: I0717 09:59:59.322971    2574 scope.go:117] "RemoveContainer" containerID="00b400539437b2ac80ed5d81036d7d759f22e7a281b9516d9efd1a852aeb47ab"
Jul 17 09:59:59.323446 ip-10-0-136-235 kubenswrapper[2574]: E0717 09:59:59.323185    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4_odh-ai-gateway-infra(5a35a09e-a71c-4768-a6f9-500e8251d517)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4" podUID="5a35a09e-a71c-4768-a6f9-500e8251d517"
Jul 17 10:00:00.147144 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:00.147117    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-10ce7fe6-29738040-b7pkf"]
Jul 17 10:00:00.152099 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:00.152076    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738040-vgggn"]
Jul 17 10:00:00.152270 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:00.152202    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-10ce7fe6-29738040-b7pkf"
Jul 17 10:00:00.155295 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:00.155281    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738040-vgggn"
Jul 17 10:00:00.164326 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:00.164308    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-10ce7fe6-29738040-b7pkf"
Jul 17 10:00:00.165037 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:00.165017    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-10ce7fe6-29738040-b7pkf"]
Jul 17 10:00:00.168138 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:00.168110    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738040-vgggn"]
Jul 17 10:00:00.172146 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:00.172128    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738040-vgggn"
Jul 17 10:00:00.321075 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:00.320924    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-10ce7fe6-29738040-b7pkf"]
Jul 17 10:00:00.323710 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:00:00.323681    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95ecdf4a_1f8a_44fd_bc5d_11d10093e84e.slice/crio-26ea8b5b37270cf7114b5ce3996e69f9de72fa980870f3392ca204f5a0e362ec WatchSource:0}: Error finding container 26ea8b5b37270cf7114b5ce3996e69f9de72fa980870f3392ca204f5a0e362ec: Status 404 returned error can't find the container with id 26ea8b5b37270cf7114b5ce3996e69f9de72fa980870f3392ca204f5a0e362ec
Jul 17 10:00:00.339485 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:00.339457    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29738040-vgggn"]
Jul 17 10:00:00.344970 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:00:00.344947    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5876f95d_d924_4f5f_acaa_8413677c0013.slice/crio-2e71899bec3515479c23561d34e53a60f1a58f6f91054df96438b90585fe82a4 WatchSource:0}: Error finding container 2e71899bec3515479c23561d34e53a60f1a58f6f91054df96438b90585fe82a4: Status 404 returned error can't find the container with id 2e71899bec3515479c23561d34e53a60f1a58f6f91054df96438b90585fe82a4
Jul 17 10:00:01.336962 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:01.336927    2574 generic.go:356] "Generic (PLEG): container finished" podID="5876f95d-d924-4f5f-acaa-8413677c0013" containerID="8987cc4a264b897504844f347ace978d40cc514748628532b0f1e4a319eee0ea" exitCode=0
Jul 17 10:00:01.337395 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:01.337006    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738040-vgggn" event={"ID":"5876f95d-d924-4f5f-acaa-8413677c0013","Type":"ContainerDied","Data":"8987cc4a264b897504844f347ace978d40cc514748628532b0f1e4a319eee0ea"}
Jul 17 10:00:01.337395 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:01.337070    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738040-vgggn" event={"ID":"5876f95d-d924-4f5f-acaa-8413677c0013","Type":"ContainerStarted","Data":"2e71899bec3515479c23561d34e53a60f1a58f6f91054df96438b90585fe82a4"}
Jul 17 10:00:01.338590 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:01.338566    2574 generic.go:356] "Generic (PLEG): container finished" podID="95ecdf4a-1f8a-44fd-bc5d-11d10093e84e" containerID="cdd65934f8e54eb2f06d8b97911e0f535a681c05fc026feaee95bb897293fd29" exitCode=7
Jul 17 10:00:01.338687 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:01.338612    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-10ce7fe6-29738040-b7pkf" event={"ID":"95ecdf4a-1f8a-44fd-bc5d-11d10093e84e","Type":"ContainerDied","Data":"cdd65934f8e54eb2f06d8b97911e0f535a681c05fc026feaee95bb897293fd29"}
Jul 17 10:00:01.338687 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:01.338655    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-10ce7fe6-29738040-b7pkf" event={"ID":"95ecdf4a-1f8a-44fd-bc5d-11d10093e84e","Type":"ContainerStarted","Data":"26ea8b5b37270cf7114b5ce3996e69f9de72fa980870f3392ca204f5a0e362ec"}
Jul 17 10:00:01.338802 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:01.338791    2574 scope.go:117] "RemoveContainer" containerID="cdd65934f8e54eb2f06d8b97911e0f535a681c05fc026feaee95bb897293fd29"
Jul 17 10:00:02.347858 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:02.347753    2574 generic.go:356] "Generic (PLEG): container finished" podID="95ecdf4a-1f8a-44fd-bc5d-11d10093e84e" containerID="0589a6096d32ae647b5652adcfe265e0fb5f87083a3384837de0eb27486f3dc1" exitCode=7
Jul 17 10:00:02.347858 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:02.347824    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-10ce7fe6-29738040-b7pkf" event={"ID":"95ecdf4a-1f8a-44fd-bc5d-11d10093e84e","Type":"ContainerDied","Data":"0589a6096d32ae647b5652adcfe265e0fb5f87083a3384837de0eb27486f3dc1"}
Jul 17 10:00:02.348368 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:02.347878    2574 scope.go:117] "RemoveContainer" containerID="cdd65934f8e54eb2f06d8b97911e0f535a681c05fc026feaee95bb897293fd29"
Jul 17 10:00:02.348368 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:02.348022    2574 scope.go:117] "RemoveContainer" containerID="0589a6096d32ae647b5652adcfe265e0fb5f87083a3384837de0eb27486f3dc1"
Jul 17 10:00:02.348368 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:00:02.348341    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-10ce7fe6-29738040-b7pkf_odh-ai-gateway-infra(95ecdf4a-1f8a-44fd-bc5d-11d10093e84e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-10ce7fe6-29738040-b7pkf" podUID="95ecdf4a-1f8a-44fd-bc5d-11d10093e84e"
Jul 17 10:00:02.491694 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:02.491673    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738040-vgggn"
Jul 17 10:00:03.355082 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:03.355053    2574 scope.go:117] "RemoveContainer" containerID="0589a6096d32ae647b5652adcfe265e0fb5f87083a3384837de0eb27486f3dc1"
Jul 17 10:00:03.355562 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:00:03.355326    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-10ce7fe6-29738040-b7pkf_odh-ai-gateway-infra(95ecdf4a-1f8a-44fd-bc5d-11d10093e84e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-10ce7fe6-29738040-b7pkf" podUID="95ecdf4a-1f8a-44fd-bc5d-11d10093e84e"
Jul 17 10:00:03.356213 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:03.356190    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738040-vgggn" event={"ID":"5876f95d-d924-4f5f-acaa-8413677c0013","Type":"ContainerDied","Data":"2e71899bec3515479c23561d34e53a60f1a58f6f91054df96438b90585fe82a4"}
Jul 17 10:00:03.356333 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:03.356231    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e71899bec3515479c23561d34e53a60f1a58f6f91054df96438b90585fe82a4"
Jul 17 10:00:03.356333 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:03.356230    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29738040-vgggn"
Jul 17 10:00:05.891430 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:05.891395    2574 scope.go:117] "RemoveContainer" containerID="04c78df9b02c16cacbdda29f8b9b44415ce7859d94711caa6824de1886f14c20"
Jul 17 10:00:06.375412 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:06.375378    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-brk6f" event={"ID":"e0d15ca8-cbc4-483d-a5f2-4eed7ad68746","Type":"ContainerStarted","Data":"c6add9b42e75dd9b83f53e04ad1378ba0330a292ff3d58b1542b88d8fba7fd30"}
Jul 17 10:00:06.394839 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:06.394787    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-brk6f" podStartSLOduration=19.394771428 podStartE2EDuration="19.394771428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:59:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:00:06.393010918 +0000 UTC m=+2003.975649314" watchObservedRunningTime="2026-07-17 10:00:06.394771428 +0000 UTC m=+2003.977409803"
Jul 17 10:00:06.440915 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:06.440882    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6c5c6b98f-rlrg6_8ab844e0-7c94-4003-9e42-ada02b05d4e9/manager/0.log"
Jul 17 10:00:07.382879 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:07.382848    2574 generic.go:356] "Generic (PLEG): container finished" podID="e0d15ca8-cbc4-483d-a5f2-4eed7ad68746" containerID="c6add9b42e75dd9b83f53e04ad1378ba0330a292ff3d58b1542b88d8fba7fd30" exitCode=6
Jul 17 10:00:07.383380 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:07.382930    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-brk6f" event={"ID":"e0d15ca8-cbc4-483d-a5f2-4eed7ad68746","Type":"ContainerDied","Data":"c6add9b42e75dd9b83f53e04ad1378ba0330a292ff3d58b1542b88d8fba7fd30"}
Jul 17 10:00:07.383380 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:07.383003    2574 scope.go:117] "RemoveContainer" containerID="04c78df9b02c16cacbdda29f8b9b44415ce7859d94711caa6824de1886f14c20"
Jul 17 10:00:07.383380 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:07.383376    2574 scope.go:117] "RemoveContainer" containerID="c6add9b42e75dd9b83f53e04ad1378ba0330a292ff3d58b1542b88d8fba7fd30"
Jul 17 10:00:07.383652 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:00:07.383631    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-brk6f_odh-ai-gateway-infra(e0d15ca8-cbc4-483d-a5f2-4eed7ad68746)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-brk6f" podUID="e0d15ca8-cbc4-483d-a5f2-4eed7ad68746"
Jul 17 10:00:07.411223 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:07.411191    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-brk6f"]
Jul 17 10:00:08.518838 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:08.518815    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-brk6f"
Jul 17 10:00:08.641977 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:08.641887    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0d15ca8-cbc4-483d-a5f2-4eed7ad68746-maas-api-service-ca\") pod \"e0d15ca8-cbc4-483d-a5f2-4eed7ad68746\" (UID: \"e0d15ca8-cbc4-483d-a5f2-4eed7ad68746\") "
Jul 17 10:00:08.642211 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:08.642184    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0d15ca8-cbc4-483d-a5f2-4eed7ad68746-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e0d15ca8-cbc4-483d-a5f2-4eed7ad68746" (UID: "e0d15ca8-cbc4-483d-a5f2-4eed7ad68746"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:00:08.742703 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:08.742666    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0d15ca8-cbc4-483d-a5f2-4eed7ad68746-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:00:09.395525 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:09.395496    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-brk6f"
Jul 17 10:00:09.395721 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:09.395537    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-brk6f" event={"ID":"e0d15ca8-cbc4-483d-a5f2-4eed7ad68746","Type":"ContainerDied","Data":"60ae973a870e7640aac8f9ebc4f0daccf620b4e534924c15f1190fa818215eaa"}
Jul 17 10:00:09.395721 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:09.395603    2574 scope.go:117] "RemoveContainer" containerID="c6add9b42e75dd9b83f53e04ad1378ba0330a292ff3d58b1542b88d8fba7fd30"
Jul 17 10:00:09.431809 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:09.431770    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-brk6f"]
Jul 17 10:00:09.444116 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:09.444082    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-brk6f"]
Jul 17 10:00:10.898529 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:10.898499    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0d15ca8-cbc4-483d-a5f2-4eed7ad68746" path="/var/lib/kubelet/pods/e0d15ca8-cbc4-483d-a5f2-4eed7ad68746/volumes"
Jul 17 10:00:12.893927 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:12.893878    2574 scope.go:117] "RemoveContainer" containerID="00b400539437b2ac80ed5d81036d7d759f22e7a281b9516d9efd1a852aeb47ab"
Jul 17 10:00:13.421110 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:13.421072    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4" event={"ID":"5a35a09e-a71c-4768-a6f9-500e8251d517","Type":"ContainerStarted","Data":"4d3e45f39ec179c79716d70a9c5ee5b4d8ba7b998d7fae71c2d45caa896e17ef"}
Jul 17 10:00:14.428716 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:14.428671    2574 generic.go:356] "Generic (PLEG): container finished" podID="5a35a09e-a71c-4768-a6f9-500e8251d517" containerID="4d3e45f39ec179c79716d70a9c5ee5b4d8ba7b998d7fae71c2d45caa896e17ef" exitCode=7
Jul 17 10:00:14.429190 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:14.428732    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4" event={"ID":"5a35a09e-a71c-4768-a6f9-500e8251d517","Type":"ContainerDied","Data":"4d3e45f39ec179c79716d70a9c5ee5b4d8ba7b998d7fae71c2d45caa896e17ef"}
Jul 17 10:00:14.429190 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:14.428793    2574 scope.go:117] "RemoveContainer" containerID="00b400539437b2ac80ed5d81036d7d759f22e7a281b9516d9efd1a852aeb47ab"
Jul 17 10:00:14.429190 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:14.429139    2574 scope.go:117] "RemoveContainer" containerID="4d3e45f39ec179c79716d70a9c5ee5b4d8ba7b998d7fae71c2d45caa896e17ef"
Jul 17 10:00:14.429425 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:00:14.429401    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4_odh-ai-gateway-infra(5a35a09e-a71c-4768-a6f9-500e8251d517)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4" podUID="5a35a09e-a71c-4768-a6f9-500e8251d517"
Jul 17 10:00:14.462093 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:14.462045    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4" podStartSLOduration=19.462032598 podStartE2EDuration="19.462032598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 09:59:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:00:13.448380985 +0000 UTC m=+2011.031019360" watchObservedRunningTime="2026-07-17 10:00:14.462032598 +0000 UTC m=+2012.044670986"
Jul 17 10:00:14.463288 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:14.463265    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4"]
Jul 17 10:00:15.578918 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:15.578894    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4"
Jul 17 10:00:15.715357 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:15.715268    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a35a09e-a71c-4768-a6f9-500e8251d517-maas-api-service-ca\") pod \"5a35a09e-a71c-4768-a6f9-500e8251d517\" (UID: \"5a35a09e-a71c-4768-a6f9-500e8251d517\") "
Jul 17 10:00:15.715668 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:15.715649    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a35a09e-a71c-4768-a6f9-500e8251d517-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5a35a09e-a71c-4768-a6f9-500e8251d517" (UID: "5a35a09e-a71c-4768-a6f9-500e8251d517"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:00:15.816339 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:15.816294    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a35a09e-a71c-4768-a6f9-500e8251d517-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:00:16.441952 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.441921    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4"
Jul 17 10:00:16.442138 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.441959    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4" event={"ID":"5a35a09e-a71c-4768-a6f9-500e8251d517","Type":"ContainerDied","Data":"685b086331ffeadc811f090ab97e45e6e8e0a2bb69165233efe5d61ad54a5e47"}
Jul 17 10:00:16.442138 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.442027    2574 scope.go:117] "RemoveContainer" containerID="4d3e45f39ec179c79716d70a9c5ee5b4d8ba7b998d7fae71c2d45caa896e17ef"
Jul 17 10:00:16.473668 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.473641    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4"]
Jul 17 10:00:16.478066 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.478042    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-gp7j4"]
Jul 17 10:00:16.587258 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.587208    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-4465d"]
Jul 17 10:00:16.587711 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.587695    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a35a09e-a71c-4768-a6f9-500e8251d517" containerName="revoke-keys"
Jul 17 10:00:16.587753 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.587713    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a35a09e-a71c-4768-a6f9-500e8251d517" containerName="revoke-keys"
Jul 17 10:00:16.587753 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.587725    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0d15ca8-cbc4-483d-a5f2-4eed7ad68746" containerName="revoke-keys"
Jul 17 10:00:16.587753 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.587731    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0d15ca8-cbc4-483d-a5f2-4eed7ad68746" containerName="revoke-keys"
Jul 17 10:00:16.587753 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.587737    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a35a09e-a71c-4768-a6f9-500e8251d517" containerName="revoke-keys"
Jul 17 10:00:16.587753 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.587743    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a35a09e-a71c-4768-a6f9-500e8251d517" containerName="revoke-keys"
Jul 17 10:00:16.587753 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.587749    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0d15ca8-cbc4-483d-a5f2-4eed7ad68746" containerName="revoke-keys"
Jul 17 10:00:16.587753 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.587755    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0d15ca8-cbc4-483d-a5f2-4eed7ad68746" containerName="revoke-keys"
Jul 17 10:00:16.587959 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.587767    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a35a09e-a71c-4768-a6f9-500e8251d517" containerName="revoke-keys"
Jul 17 10:00:16.587959 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.587773    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a35a09e-a71c-4768-a6f9-500e8251d517" containerName="revoke-keys"
Jul 17 10:00:16.587959 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.587788    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5876f95d-d924-4f5f-acaa-8413677c0013" containerName="cleanup"
Jul 17 10:00:16.587959 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.587793    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5876f95d-d924-4f5f-acaa-8413677c0013" containerName="cleanup"
Jul 17 10:00:16.587959 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.587851    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0d15ca8-cbc4-483d-a5f2-4eed7ad68746" containerName="revoke-keys"
Jul 17 10:00:16.587959 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.587859    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0d15ca8-cbc4-483d-a5f2-4eed7ad68746" containerName="revoke-keys"
Jul 17 10:00:16.587959 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.587865    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a35a09e-a71c-4768-a6f9-500e8251d517" containerName="revoke-keys"
Jul 17 10:00:16.587959 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.587871    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a35a09e-a71c-4768-a6f9-500e8251d517" containerName="revoke-keys"
Jul 17 10:00:16.587959 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.587876    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5876f95d-d924-4f5f-acaa-8413677c0013" containerName="cleanup"
Jul 17 10:00:16.591959 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.591942    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-4465d"
Jul 17 10:00:16.609225 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.609190    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-4465d"]
Jul 17 10:00:16.724075 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.723978    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c36f1f08-4f4c-4933-84ce-87a71671e7e3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-4465d\" (UID: \"c36f1f08-4f4c-4933-84ce-87a71671e7e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-4465d"
Jul 17 10:00:16.824887 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.824853    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c36f1f08-4f4c-4933-84ce-87a71671e7e3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-4465d\" (UID: \"c36f1f08-4f4c-4933-84ce-87a71671e7e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-4465d"
Jul 17 10:00:16.825510 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.825491    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c36f1f08-4f4c-4933-84ce-87a71671e7e3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-4465d\" (UID: \"c36f1f08-4f4c-4933-84ce-87a71671e7e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-4465d"
Jul 17 10:00:16.892131 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.892098    2574 scope.go:117] "RemoveContainer" containerID="0589a6096d32ae647b5652adcfe265e0fb5f87083a3384837de0eb27486f3dc1"
Jul 17 10:00:16.898526 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.898499    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a35a09e-a71c-4768-a6f9-500e8251d517" path="/var/lib/kubelet/pods/5a35a09e-a71c-4768-a6f9-500e8251d517/volumes"
Jul 17 10:00:16.902571 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:16.902549    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-4465d"
Jul 17 10:00:17.282222 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:17.282195    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-4465d"]
Jul 17 10:00:17.284479 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:00:17.284446    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc36f1f08_4f4c_4933_84ce_87a71671e7e3.slice/crio-b802764afd06b244b61a97c9d311067aa77c9b7a3bb2a1b4ba930926d7b3bb72 WatchSource:0}: Error finding container b802764afd06b244b61a97c9d311067aa77c9b7a3bb2a1b4ba930926d7b3bb72: Status 404 returned error can't find the container with id b802764afd06b244b61a97c9d311067aa77c9b7a3bb2a1b4ba930926d7b3bb72
Jul 17 10:00:17.448671 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:17.448628    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-4465d" event={"ID":"c36f1f08-4f4c-4933-84ce-87a71671e7e3","Type":"ContainerStarted","Data":"b802764afd06b244b61a97c9d311067aa77c9b7a3bb2a1b4ba930926d7b3bb72"}
Jul 17 10:00:17.451121 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:17.451100    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-10ce7fe6-29738040-b7pkf" event={"ID":"95ecdf4a-1f8a-44fd-bc5d-11d10093e84e","Type":"ContainerStarted","Data":"251265bcdd1b456f2407ae147dd65a91b36d3f5213793bdbb359bab319c5b5ff"}
Jul 17 10:00:18.458548 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:18.458514    2574 generic.go:356] "Generic (PLEG): container finished" podID="95ecdf4a-1f8a-44fd-bc5d-11d10093e84e" containerID="251265bcdd1b456f2407ae147dd65a91b36d3f5213793bdbb359bab319c5b5ff" exitCode=7
Jul 17 10:00:18.458974 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:18.458589    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-10ce7fe6-29738040-b7pkf" event={"ID":"95ecdf4a-1f8a-44fd-bc5d-11d10093e84e","Type":"ContainerDied","Data":"251265bcdd1b456f2407ae147dd65a91b36d3f5213793bdbb359bab319c5b5ff"}
Jul 17 10:00:18.458974 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:18.458651    2574 scope.go:117] "RemoveContainer" containerID="0589a6096d32ae647b5652adcfe265e0fb5f87083a3384837de0eb27486f3dc1"
Jul 17 10:00:18.458974 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:18.458878    2574 scope.go:117] "RemoveContainer" containerID="251265bcdd1b456f2407ae147dd65a91b36d3f5213793bdbb359bab319c5b5ff"
Jul 17 10:00:18.459160 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:00:18.459122    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-ait-10ce7fe6-29738040-b7pkf_odh-ai-gateway-infra(95ecdf4a-1f8a-44fd-bc5d-11d10093e84e)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-10ce7fe6-29738040-b7pkf" podUID="95ecdf4a-1f8a-44fd-bc5d-11d10093e84e"
Jul 17 10:00:18.460341 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:18.460315    2574 generic.go:356] "Generic (PLEG): container finished" podID="c36f1f08-4f4c-4933-84ce-87a71671e7e3" containerID="1a321b44202e39967e3a6db4d9b4324096f3046161f7b44f90413ac640a0757b" exitCode=6
Jul 17 10:00:18.460447 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:18.460397    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-4465d" event={"ID":"c36f1f08-4f4c-4933-84ce-87a71671e7e3","Type":"ContainerDied","Data":"1a321b44202e39967e3a6db4d9b4324096f3046161f7b44f90413ac640a0757b"}
Jul 17 10:00:18.460535 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:18.460522    2574 scope.go:117] "RemoveContainer" containerID="1a321b44202e39967e3a6db4d9b4324096f3046161f7b44f90413ac640a0757b"
Jul 17 10:00:18.497286 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:18.497225    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-10ce7fe6-29738040-b7pkf" podStartSLOduration=18.497207821 podStartE2EDuration="18.497207821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:00:17.477141028 +0000 UTC m=+2015.059779421" watchObservedRunningTime="2026-07-17 10:00:18.497207821 +0000 UTC m=+2016.079846196"
Jul 17 10:00:18.497932 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:18.497911    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-10ce7fe6-29738040-b7pkf"]
Jul 17 10:00:19.466917 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:19.466884    2574 generic.go:356] "Generic (PLEG): container finished" podID="c36f1f08-4f4c-4933-84ce-87a71671e7e3" containerID="20a7e1a4366eb23b84015c49883355f68b0c9308f2ba1c917ea83d2305a4ff82" exitCode=6
Jul 17 10:00:19.467413 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:19.467037    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-4465d" event={"ID":"c36f1f08-4f4c-4933-84ce-87a71671e7e3","Type":"ContainerDied","Data":"20a7e1a4366eb23b84015c49883355f68b0c9308f2ba1c917ea83d2305a4ff82"}
Jul 17 10:00:19.467413 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:19.467101    2574 scope.go:117] "RemoveContainer" containerID="1a321b44202e39967e3a6db4d9b4324096f3046161f7b44f90413ac640a0757b"
Jul 17 10:00:19.467413 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:19.467232    2574 scope.go:117] "RemoveContainer" containerID="20a7e1a4366eb23b84015c49883355f68b0c9308f2ba1c917ea83d2305a4ff82"
Jul 17 10:00:19.467563 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:00:19.467495    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-4465d_odh-ai-gateway-infra(c36f1f08-4f4c-4933-84ce-87a71671e7e3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-4465d" podUID="c36f1f08-4f4c-4933-84ce-87a71671e7e3"
Jul 17 10:00:19.608179 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:19.608153    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-10ce7fe6-29738040-b7pkf"
Jul 17 10:00:20.474995 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:20.474962    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-10ce7fe6-29738040-b7pkf"
Jul 17 10:00:20.475482 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:20.475009    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-10ce7fe6-29738040-b7pkf" event={"ID":"95ecdf4a-1f8a-44fd-bc5d-11d10093e84e","Type":"ContainerDied","Data":"26ea8b5b37270cf7114b5ce3996e69f9de72fa980870f3392ca204f5a0e362ec"}
Jul 17 10:00:20.475482 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:20.475084    2574 scope.go:117] "RemoveContainer" containerID="251265bcdd1b456f2407ae147dd65a91b36d3f5213793bdbb359bab319c5b5ff"
Jul 17 10:00:20.476852 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:20.476836    2574 scope.go:117] "RemoveContainer" containerID="20a7e1a4366eb23b84015c49883355f68b0c9308f2ba1c917ea83d2305a4ff82"
Jul 17 10:00:20.477046 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:00:20.477026    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-4465d_odh-ai-gateway-infra(c36f1f08-4f4c-4933-84ce-87a71671e7e3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-4465d" podUID="c36f1f08-4f4c-4933-84ce-87a71671e7e3"
Jul 17 10:00:20.539505 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:20.539464    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-10ce7fe6-29738040-b7pkf"]
Jul 17 10:00:20.544345 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:20.544314    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-10ce7fe6-29738040-b7pkf"]
Jul 17 10:00:20.897840 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:20.897740    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95ecdf4a-1f8a-44fd-bc5d-11d10093e84e" path="/var/lib/kubelet/pods/95ecdf4a-1f8a-44fd-bc5d-11d10093e84e/volumes"
Jul 17 10:00:32.898693 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:32.898661    2574 scope.go:117] "RemoveContainer" containerID="20a7e1a4366eb23b84015c49883355f68b0c9308f2ba1c917ea83d2305a4ff82"
Jul 17 10:00:33.545101 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:33.545071    2574 generic.go:356] "Generic (PLEG): container finished" podID="c36f1f08-4f4c-4933-84ce-87a71671e7e3" containerID="8331a986c5a47b5ad203b4e4432c406bdf37b68fae6ce386b61bf3da80ab8236" exitCode=6
Jul 17 10:00:33.545257 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:33.545148    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-4465d" event={"ID":"c36f1f08-4f4c-4933-84ce-87a71671e7e3","Type":"ContainerDied","Data":"8331a986c5a47b5ad203b4e4432c406bdf37b68fae6ce386b61bf3da80ab8236"}
Jul 17 10:00:33.545257 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:33.545218    2574 scope.go:117] "RemoveContainer" containerID="20a7e1a4366eb23b84015c49883355f68b0c9308f2ba1c917ea83d2305a4ff82"
Jul 17 10:00:33.545501 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:33.545487    2574 scope.go:117] "RemoveContainer" containerID="8331a986c5a47b5ad203b4e4432c406bdf37b68fae6ce386b61bf3da80ab8236"
Jul 17 10:00:33.545780 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:00:33.545752    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-4465d_odh-ai-gateway-infra(c36f1f08-4f4c-4933-84ce-87a71671e7e3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-4465d" podUID="c36f1f08-4f4c-4933-84ce-87a71671e7e3"
Jul 17 10:00:34.613834 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:34.613797    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-4465d"]
Jul 17 10:00:34.752188 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:34.752153    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-4465d"
Jul 17 10:00:34.789441 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:34.789414    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c36f1f08-4f4c-4933-84ce-87a71671e7e3-maas-api-service-ca\") pod \"c36f1f08-4f4c-4933-84ce-87a71671e7e3\" (UID: \"c36f1f08-4f4c-4933-84ce-87a71671e7e3\") "
Jul 17 10:00:34.789814 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:34.789786    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c36f1f08-4f4c-4933-84ce-87a71671e7e3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c36f1f08-4f4c-4933-84ce-87a71671e7e3" (UID: "c36f1f08-4f4c-4933-84ce-87a71671e7e3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:00:34.890743 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:34.890661    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c36f1f08-4f4c-4933-84ce-87a71671e7e3-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:00:35.558924 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:35.558896    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-4465d"
Jul 17 10:00:35.559100 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:35.558885    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-4465d" event={"ID":"c36f1f08-4f4c-4933-84ce-87a71671e7e3","Type":"ContainerDied","Data":"b802764afd06b244b61a97c9d311067aa77c9b7a3bb2a1b4ba930926d7b3bb72"}
Jul 17 10:00:35.559100 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:35.559039    2574 scope.go:117] "RemoveContainer" containerID="8331a986c5a47b5ad203b4e4432c406bdf37b68fae6ce386b61bf3da80ab8236"
Jul 17 10:00:35.593250 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:35.593218    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-4465d"]
Jul 17 10:00:35.605755 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:35.605727    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-4465d"]
Jul 17 10:00:36.897417 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:36.897386    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c36f1f08-4f4c-4933-84ce-87a71671e7e3" path="/var/lib/kubelet/pods/c36f1f08-4f4c-4933-84ce-87a71671e7e3/volumes"
Jul 17 10:00:47.808064 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:47.808026    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-54z92"]
Jul 17 10:00:47.808509 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:47.808493    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95ecdf4a-1f8a-44fd-bc5d-11d10093e84e" containerName="cleanup"
Jul 17 10:00:47.808554 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:47.808511    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="95ecdf4a-1f8a-44fd-bc5d-11d10093e84e" containerName="cleanup"
Jul 17 10:00:47.808554 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:47.808527    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0d15ca8-cbc4-483d-a5f2-4eed7ad68746" containerName="revoke-keys"
Jul 17 10:00:47.808554 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:47.808532    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0d15ca8-cbc4-483d-a5f2-4eed7ad68746" containerName="revoke-keys"
Jul 17 10:00:47.808554 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:47.808541    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c36f1f08-4f4c-4933-84ce-87a71671e7e3" containerName="revoke-keys"
Jul 17 10:00:47.808554 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:47.808547    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c36f1f08-4f4c-4933-84ce-87a71671e7e3" containerName="revoke-keys"
Jul 17 10:00:47.808711 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:47.808565    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95ecdf4a-1f8a-44fd-bc5d-11d10093e84e" containerName="cleanup"
Jul 17 10:00:47.808711 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:47.808570    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="95ecdf4a-1f8a-44fd-bc5d-11d10093e84e" containerName="cleanup"
Jul 17 10:00:47.808711 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:47.808579    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c36f1f08-4f4c-4933-84ce-87a71671e7e3" containerName="revoke-keys"
Jul 17 10:00:47.808711 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:47.808584    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c36f1f08-4f4c-4933-84ce-87a71671e7e3" containerName="revoke-keys"
Jul 17 10:00:47.808711 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:47.808640    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c36f1f08-4f4c-4933-84ce-87a71671e7e3" containerName="revoke-keys"
Jul 17 10:00:47.808711 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:47.808647    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0d15ca8-cbc4-483d-a5f2-4eed7ad68746" containerName="revoke-keys"
Jul 17 10:00:47.808711 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:47.808653    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c36f1f08-4f4c-4933-84ce-87a71671e7e3" containerName="revoke-keys"
Jul 17 10:00:47.808711 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:47.808659    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="95ecdf4a-1f8a-44fd-bc5d-11d10093e84e" containerName="cleanup"
Jul 17 10:00:47.808711 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:47.808665    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a35a09e-a71c-4768-a6f9-500e8251d517" containerName="revoke-keys"
Jul 17 10:00:47.808711 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:47.808670    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="95ecdf4a-1f8a-44fd-bc5d-11d10093e84e" containerName="cleanup"
Jul 17 10:00:47.808711 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:47.808676    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="95ecdf4a-1f8a-44fd-bc5d-11d10093e84e" containerName="cleanup"
Jul 17 10:00:47.811863 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:47.811845    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-54z92"
Jul 17 10:00:47.825527 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:47.825507    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:00:47.851010 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:47.850927    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-54z92"]
Jul 17 10:00:47.921133 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:47.921102    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08f171ec-363b-44e3-af40-8daaf2a700be-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-54z92\" (UID: \"08f171ec-363b-44e3-af40-8daaf2a700be\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-54z92"
Jul 17 10:00:48.022564 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:48.022518    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08f171ec-363b-44e3-af40-8daaf2a700be-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-54z92\" (UID: \"08f171ec-363b-44e3-af40-8daaf2a700be\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-54z92"
Jul 17 10:00:48.023183 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:48.023156    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08f171ec-363b-44e3-af40-8daaf2a700be-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-54z92\" (UID: \"08f171ec-363b-44e3-af40-8daaf2a700be\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-54z92"
Jul 17 10:00:48.121703 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:48.121610    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-54z92"
Jul 17 10:00:48.269508 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:48.269475    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-54z92"]
Jul 17 10:00:48.273407 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:00:48.273362    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08f171ec_363b_44e3_af40_8daaf2a700be.slice/crio-2bd1207dacea23285c2a7a294cdbf8c7c0a93e22b03926a2a5f3584c2da07a55 WatchSource:0}: Error finding container 2bd1207dacea23285c2a7a294cdbf8c7c0a93e22b03926a2a5f3584c2da07a55: Status 404 returned error can't find the container with id 2bd1207dacea23285c2a7a294cdbf8c7c0a93e22b03926a2a5f3584c2da07a55
Jul 17 10:00:48.633275 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:48.633222    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-54z92" event={"ID":"08f171ec-363b-44e3-af40-8daaf2a700be","Type":"ContainerStarted","Data":"e3a72b2d674986d0d48e7d5e7bd8478fdb777d3ed1cc77daa61e9d7e3c752add"}
Jul 17 10:00:48.633453 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:48.633299    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-54z92" event={"ID":"08f171ec-363b-44e3-af40-8daaf2a700be","Type":"ContainerStarted","Data":"2bd1207dacea23285c2a7a294cdbf8c7c0a93e22b03926a2a5f3584c2da07a55"}
Jul 17 10:00:49.639322 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:49.639283    2574 generic.go:356] "Generic (PLEG): container finished" podID="08f171ec-363b-44e3-af40-8daaf2a700be" containerID="e3a72b2d674986d0d48e7d5e7bd8478fdb777d3ed1cc77daa61e9d7e3c752add" exitCode=6
Jul 17 10:00:49.639700 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:49.639373    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-54z92" event={"ID":"08f171ec-363b-44e3-af40-8daaf2a700be","Type":"ContainerDied","Data":"e3a72b2d674986d0d48e7d5e7bd8478fdb777d3ed1cc77daa61e9d7e3c752add"}
Jul 17 10:00:49.639700 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:49.639550    2574 scope.go:117] "RemoveContainer" containerID="e3a72b2d674986d0d48e7d5e7bd8478fdb777d3ed1cc77daa61e9d7e3c752add"
Jul 17 10:00:50.646855 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:50.646759    2574 generic.go:356] "Generic (PLEG): container finished" podID="08f171ec-363b-44e3-af40-8daaf2a700be" containerID="84e36b554728152365315b7765159c37652c3fbe89f7cd66c3992fb3714933a2" exitCode=6
Jul 17 10:00:50.646855 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:50.646798    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-54z92" event={"ID":"08f171ec-363b-44e3-af40-8daaf2a700be","Type":"ContainerDied","Data":"84e36b554728152365315b7765159c37652c3fbe89f7cd66c3992fb3714933a2"}
Jul 17 10:00:50.647338 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:50.646899    2574 scope.go:117] "RemoveContainer" containerID="e3a72b2d674986d0d48e7d5e7bd8478fdb777d3ed1cc77daa61e9d7e3c752add"
Jul 17 10:00:50.647338 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:50.647085    2574 scope.go:117] "RemoveContainer" containerID="84e36b554728152365315b7765159c37652c3fbe89f7cd66c3992fb3714933a2"
Jul 17 10:00:50.647338 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:00:50.647327    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-54z92_odh-ai-gateway-infra(08f171ec-363b-44e3-af40-8daaf2a700be)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-54z92" podUID="08f171ec-363b-44e3-af40-8daaf2a700be"
Jul 17 10:00:51.654045 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:51.654019    2574 scope.go:117] "RemoveContainer" containerID="84e36b554728152365315b7765159c37652c3fbe89f7cd66c3992fb3714933a2"
Jul 17 10:00:51.654458 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:00:51.654234    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-54z92_odh-ai-gateway-infra(08f171ec-363b-44e3-af40-8daaf2a700be)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-54z92" podUID="08f171ec-363b-44e3-af40-8daaf2a700be"
Jul 17 10:00:55.731570 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:55.731536    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97"]
Jul 17 10:00:55.732020 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:55.731979    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95ecdf4a-1f8a-44fd-bc5d-11d10093e84e" containerName="cleanup"
Jul 17 10:00:55.732020 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:55.731990    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="95ecdf4a-1f8a-44fd-bc5d-11d10093e84e" containerName="cleanup"
Jul 17 10:00:55.732020 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:55.732003    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c36f1f08-4f4c-4933-84ce-87a71671e7e3" containerName="revoke-keys"
Jul 17 10:00:55.732020 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:55.732008    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c36f1f08-4f4c-4933-84ce-87a71671e7e3" containerName="revoke-keys"
Jul 17 10:00:55.732154 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:55.732102    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c36f1f08-4f4c-4933-84ce-87a71671e7e3" containerName="revoke-keys"
Jul 17 10:00:55.736730 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:55.736713    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97"
Jul 17 10:00:55.764348 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:55.764319    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97"]
Jul 17 10:00:55.897607 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:55.897577    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d15fd3b-d5bb-46c4-9d9b-96f231782e98-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97\" (UID: \"3d15fd3b-d5bb-46c4-9d9b-96f231782e98\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97"
Jul 17 10:00:55.998702 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:55.998610    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d15fd3b-d5bb-46c4-9d9b-96f231782e98-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97\" (UID: \"3d15fd3b-d5bb-46c4-9d9b-96f231782e98\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97"
Jul 17 10:00:55.999233 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:55.999213    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d15fd3b-d5bb-46c4-9d9b-96f231782e98-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97\" (UID: \"3d15fd3b-d5bb-46c4-9d9b-96f231782e98\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97"
Jul 17 10:00:56.046641 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:56.046605    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97"
Jul 17 10:00:56.217445 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:56.217410    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97"]
Jul 17 10:00:56.218704 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:00:56.218678    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d15fd3b_d5bb_46c4_9d9b_96f231782e98.slice/crio-1b1d0efdc1d120cbf38efef49ccad86f22435ebbbd49b752691535d4fca97d91 WatchSource:0}: Error finding container 1b1d0efdc1d120cbf38efef49ccad86f22435ebbbd49b752691535d4fca97d91: Status 404 returned error can't find the container with id 1b1d0efdc1d120cbf38efef49ccad86f22435ebbbd49b752691535d4fca97d91
Jul 17 10:00:56.681644 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:56.681607    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97" event={"ID":"3d15fd3b-d5bb-46c4-9d9b-96f231782e98","Type":"ContainerStarted","Data":"2e25a26500b3f1ddae5cc9590b2467efec49463cae455cc6070d007ed29501c3"}
Jul 17 10:00:56.681848 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:56.681695    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97" event={"ID":"3d15fd3b-d5bb-46c4-9d9b-96f231782e98","Type":"ContainerStarted","Data":"1b1d0efdc1d120cbf38efef49ccad86f22435ebbbd49b752691535d4fca97d91"}
Jul 17 10:00:56.723766 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:56.723718    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97" podStartSLOduration=1.723702146 podStartE2EDuration="1.723702146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:00:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:00:56.721389238 +0000 UTC m=+2054.304027612" watchObservedRunningTime="2026-07-17 10:00:56.723702146 +0000 UTC m=+2054.306340521"
Jul 17 10:00:57.688431 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:57.688398    2574 generic.go:356] "Generic (PLEG): container finished" podID="3d15fd3b-d5bb-46c4-9d9b-96f231782e98" containerID="2e25a26500b3f1ddae5cc9590b2467efec49463cae455cc6070d007ed29501c3" exitCode=7
Jul 17 10:00:57.688431 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:57.688434    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97" event={"ID":"3d15fd3b-d5bb-46c4-9d9b-96f231782e98","Type":"ContainerDied","Data":"2e25a26500b3f1ddae5cc9590b2467efec49463cae455cc6070d007ed29501c3"}
Jul 17 10:00:57.688934 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:57.688695    2574 scope.go:117] "RemoveContainer" containerID="2e25a26500b3f1ddae5cc9590b2467efec49463cae455cc6070d007ed29501c3"
Jul 17 10:00:58.695324 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:58.695290    2574 generic.go:356] "Generic (PLEG): container finished" podID="3d15fd3b-d5bb-46c4-9d9b-96f231782e98" containerID="3870a225ec91fe7044e4b8c5aa18b335b884d368b9ce971c7b8b90d364c8a332" exitCode=7
Jul 17 10:00:58.695800 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:58.695372    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97" event={"ID":"3d15fd3b-d5bb-46c4-9d9b-96f231782e98","Type":"ContainerDied","Data":"3870a225ec91fe7044e4b8c5aa18b335b884d368b9ce971c7b8b90d364c8a332"}
Jul 17 10:00:58.695800 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:58.695431    2574 scope.go:117] "RemoveContainer" containerID="2e25a26500b3f1ddae5cc9590b2467efec49463cae455cc6070d007ed29501c3"
Jul 17 10:00:58.695800 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:58.695588    2574 scope.go:117] "RemoveContainer" containerID="3870a225ec91fe7044e4b8c5aa18b335b884d368b9ce971c7b8b90d364c8a332"
Jul 17 10:00:58.695962 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:00:58.695888    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97_odh-ai-gateway-infra(3d15fd3b-d5bb-46c4-9d9b-96f231782e98)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97" podUID="3d15fd3b-d5bb-46c4-9d9b-96f231782e98"
Jul 17 10:00:59.702483 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:00:59.702451    2574 scope.go:117] "RemoveContainer" containerID="3870a225ec91fe7044e4b8c5aa18b335b884d368b9ce971c7b8b90d364c8a332"
Jul 17 10:00:59.702884 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:00:59.702720    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97_odh-ai-gateway-infra(3d15fd3b-d5bb-46c4-9d9b-96f231782e98)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97" podUID="3d15fd3b-d5bb-46c4-9d9b-96f231782e98"
Jul 17 10:01:03.891291 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:03.891235    2574 scope.go:117] "RemoveContainer" containerID="84e36b554728152365315b7765159c37652c3fbe89f7cd66c3992fb3714933a2"
Jul 17 10:01:04.731107 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:04.731070    2574 generic.go:356] "Generic (PLEG): container finished" podID="08f171ec-363b-44e3-af40-8daaf2a700be" containerID="32298050ea25c622ca4195bb05560addd12d3136e8727f42324171dbd68b1a80" exitCode=6
Jul 17 10:01:04.731302 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:04.731143    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-54z92" event={"ID":"08f171ec-363b-44e3-af40-8daaf2a700be","Type":"ContainerDied","Data":"32298050ea25c622ca4195bb05560addd12d3136e8727f42324171dbd68b1a80"}
Jul 17 10:01:04.731302 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:04.731212    2574 scope.go:117] "RemoveContainer" containerID="84e36b554728152365315b7765159c37652c3fbe89f7cd66c3992fb3714933a2"
Jul 17 10:01:04.731518 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:04.731499    2574 scope.go:117] "RemoveContainer" containerID="32298050ea25c622ca4195bb05560addd12d3136e8727f42324171dbd68b1a80"
Jul 17 10:01:04.731765 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:01:04.731742    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-54z92_odh-ai-gateway-infra(08f171ec-363b-44e3-af40-8daaf2a700be)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-54z92" podUID="08f171ec-363b-44e3-af40-8daaf2a700be"
Jul 17 10:01:05.768901 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:05.768868    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-54z92"]
Jul 17 10:01:05.915085 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:05.915062    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-54z92"
Jul 17 10:01:05.983823 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:05.983784    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08f171ec-363b-44e3-af40-8daaf2a700be-maas-api-service-ca\") pod \"08f171ec-363b-44e3-af40-8daaf2a700be\" (UID: \"08f171ec-363b-44e3-af40-8daaf2a700be\") "
Jul 17 10:01:05.984149 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:05.984128    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08f171ec-363b-44e3-af40-8daaf2a700be-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "08f171ec-363b-44e3-af40-8daaf2a700be" (UID: "08f171ec-363b-44e3-af40-8daaf2a700be"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:01:06.084754 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:06.084674    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/08f171ec-363b-44e3-af40-8daaf2a700be-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:01:06.645514 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:06.645472    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2crrz"]
Jul 17 10:01:06.645973 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:06.645943    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08f171ec-363b-44e3-af40-8daaf2a700be" containerName="revoke-keys"
Jul 17 10:01:06.645973 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:06.645964    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="08f171ec-363b-44e3-af40-8daaf2a700be" containerName="revoke-keys"
Jul 17 10:01:06.645973 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:06.645979    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08f171ec-363b-44e3-af40-8daaf2a700be" containerName="revoke-keys"
Jul 17 10:01:06.646161 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:06.645986    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="08f171ec-363b-44e3-af40-8daaf2a700be" containerName="revoke-keys"
Jul 17 10:01:06.646161 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:06.645995    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08f171ec-363b-44e3-af40-8daaf2a700be" containerName="revoke-keys"
Jul 17 10:01:06.646161 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:06.646001    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="08f171ec-363b-44e3-af40-8daaf2a700be" containerName="revoke-keys"
Jul 17 10:01:06.646161 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:06.646083    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="08f171ec-363b-44e3-af40-8daaf2a700be" containerName="revoke-keys"
Jul 17 10:01:06.646161 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:06.646092    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="08f171ec-363b-44e3-af40-8daaf2a700be" containerName="revoke-keys"
Jul 17 10:01:06.646161 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:06.646098    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="08f171ec-363b-44e3-af40-8daaf2a700be" containerName="revoke-keys"
Jul 17 10:01:06.649163 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:06.649148    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2crrz"
Jul 17 10:01:06.657949 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:06.657924    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2crrz"]
Jul 17 10:01:06.690601 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:06.690571    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7914e85-46bd-49b9-9567-187f0dc2d576-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-2crrz\" (UID: \"c7914e85-46bd-49b9-9567-187f0dc2d576\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2crrz"
Jul 17 10:01:06.743709 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:06.743683    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-54z92"
Jul 17 10:01:06.743875 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:06.743676    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-54z92" event={"ID":"08f171ec-363b-44e3-af40-8daaf2a700be","Type":"ContainerDied","Data":"2bd1207dacea23285c2a7a294cdbf8c7c0a93e22b03926a2a5f3584c2da07a55"}
Jul 17 10:01:06.743875 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:06.743845    2574 scope.go:117] "RemoveContainer" containerID="32298050ea25c622ca4195bb05560addd12d3136e8727f42324171dbd68b1a80"
Jul 17 10:01:06.768813 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:06.768779    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-54z92"]
Jul 17 10:01:06.774975 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:06.774944    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-54z92"]
Jul 17 10:01:06.791964 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:06.791934    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7914e85-46bd-49b9-9567-187f0dc2d576-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-2crrz\" (UID: \"c7914e85-46bd-49b9-9567-187f0dc2d576\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2crrz"
Jul 17 10:01:06.792554 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:06.792535    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7914e85-46bd-49b9-9567-187f0dc2d576-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-2crrz\" (UID: \"c7914e85-46bd-49b9-9567-187f0dc2d576\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2crrz"
Jul 17 10:01:06.897313 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:06.897197    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08f171ec-363b-44e3-af40-8daaf2a700be" path="/var/lib/kubelet/pods/08f171ec-363b-44e3-af40-8daaf2a700be/volumes"
Jul 17 10:01:06.960365 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:06.960325    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2crrz"
Jul 17 10:01:07.102804 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:07.102775    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2crrz"]
Jul 17 10:01:07.104792 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:01:07.104764    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7914e85_46bd_49b9_9567_187f0dc2d576.slice/crio-6d45a630f0b99b86abbfae80ab63b98eaf30aa05e26dc3724024394a5614a581 WatchSource:0}: Error finding container 6d45a630f0b99b86abbfae80ab63b98eaf30aa05e26dc3724024394a5614a581: Status 404 returned error can't find the container with id 6d45a630f0b99b86abbfae80ab63b98eaf30aa05e26dc3724024394a5614a581
Jul 17 10:01:07.751631 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:07.751594    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2crrz" event={"ID":"c7914e85-46bd-49b9-9567-187f0dc2d576","Type":"ContainerStarted","Data":"c24f3c0e23cb1466af8c0f493f0f7f4394f3dc47af2e15e60d2a690aad4fd239"}
Jul 17 10:01:07.751822 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:07.751660    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2crrz" event={"ID":"c7914e85-46bd-49b9-9567-187f0dc2d576","Type":"ContainerStarted","Data":"6d45a630f0b99b86abbfae80ab63b98eaf30aa05e26dc3724024394a5614a581"}
Jul 17 10:01:07.772322 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:07.772274    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2crrz" podStartSLOduration=1.77226016 podStartE2EDuration="1.77226016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:01:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:01:07.769390495 +0000 UTC m=+2065.352028872" watchObservedRunningTime="2026-07-17 10:01:07.77226016 +0000 UTC m=+2065.354898534"
Jul 17 10:01:08.758091 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:08.758059    2574 generic.go:356] "Generic (PLEG): container finished" podID="c7914e85-46bd-49b9-9567-187f0dc2d576" containerID="c24f3c0e23cb1466af8c0f493f0f7f4394f3dc47af2e15e60d2a690aad4fd239" exitCode=6
Jul 17 10:01:08.758506 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:08.758178    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2crrz" event={"ID":"c7914e85-46bd-49b9-9567-187f0dc2d576","Type":"ContainerDied","Data":"c24f3c0e23cb1466af8c0f493f0f7f4394f3dc47af2e15e60d2a690aad4fd239"}
Jul 17 10:01:08.758506 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:08.758417    2574 scope.go:117] "RemoveContainer" containerID="c24f3c0e23cb1466af8c0f493f0f7f4394f3dc47af2e15e60d2a690aad4fd239"
Jul 17 10:01:09.765076 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:09.765035    2574 generic.go:356] "Generic (PLEG): container finished" podID="c7914e85-46bd-49b9-9567-187f0dc2d576" containerID="c9e218f2618614b2c3ba0e7fd9b2895354b62d3534a4d121e37648c86fb6a399" exitCode=6
Jul 17 10:01:09.765531 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:09.765090    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2crrz" event={"ID":"c7914e85-46bd-49b9-9567-187f0dc2d576","Type":"ContainerDied","Data":"c9e218f2618614b2c3ba0e7fd9b2895354b62d3534a4d121e37648c86fb6a399"}
Jul 17 10:01:09.765531 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:09.765154    2574 scope.go:117] "RemoveContainer" containerID="c24f3c0e23cb1466af8c0f493f0f7f4394f3dc47af2e15e60d2a690aad4fd239"
Jul 17 10:01:09.765531 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:09.765369    2574 scope.go:117] "RemoveContainer" containerID="c9e218f2618614b2c3ba0e7fd9b2895354b62d3534a4d121e37648c86fb6a399"
Jul 17 10:01:09.765668 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:01:09.765593    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-2crrz_odh-ai-gateway-infra(c7914e85-46bd-49b9-9567-187f0dc2d576)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2crrz" podUID="c7914e85-46bd-49b9-9567-187f0dc2d576"
Jul 17 10:01:10.772126 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:10.772099    2574 scope.go:117] "RemoveContainer" containerID="c9e218f2618614b2c3ba0e7fd9b2895354b62d3534a4d121e37648c86fb6a399"
Jul 17 10:01:10.772505 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:01:10.772361    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-2crrz_odh-ai-gateway-infra(c7914e85-46bd-49b9-9567-187f0dc2d576)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2crrz" podUID="c7914e85-46bd-49b9-9567-187f0dc2d576"
Jul 17 10:01:14.891629 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:14.891596    2574 scope.go:117] "RemoveContainer" containerID="3870a225ec91fe7044e4b8c5aa18b335b884d368b9ce971c7b8b90d364c8a332"
Jul 17 10:01:15.799754 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:15.799676    2574 generic.go:356] "Generic (PLEG): container finished" podID="3d15fd3b-d5bb-46c4-9d9b-96f231782e98" containerID="dcb07f769f3b4cacd6276316e2ea3a169e26c1d8c1fe3ad9308179c2eaaaa194" exitCode=7
Jul 17 10:01:15.799897 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:15.799753    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97" event={"ID":"3d15fd3b-d5bb-46c4-9d9b-96f231782e98","Type":"ContainerDied","Data":"dcb07f769f3b4cacd6276316e2ea3a169e26c1d8c1fe3ad9308179c2eaaaa194"}
Jul 17 10:01:15.799897 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:15.799817    2574 scope.go:117] "RemoveContainer" containerID="3870a225ec91fe7044e4b8c5aa18b335b884d368b9ce971c7b8b90d364c8a332"
Jul 17 10:01:15.800168 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:15.800152    2574 scope.go:117] "RemoveContainer" containerID="dcb07f769f3b4cacd6276316e2ea3a169e26c1d8c1fe3ad9308179c2eaaaa194"
Jul 17 10:01:15.800450 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:01:15.800433    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97_odh-ai-gateway-infra(3d15fd3b-d5bb-46c4-9d9b-96f231782e98)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97" podUID="3d15fd3b-d5bb-46c4-9d9b-96f231782e98"
Jul 17 10:01:16.838101 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:16.838067    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97"]
Jul 17 10:01:16.966552 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:16.966531    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97"
Jul 17 10:01:17.087716 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:17.087683    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d15fd3b-d5bb-46c4-9d9b-96f231782e98-maas-api-service-ca\") pod \"3d15fd3b-d5bb-46c4-9d9b-96f231782e98\" (UID: \"3d15fd3b-d5bb-46c4-9d9b-96f231782e98\") "
Jul 17 10:01:17.088066 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:17.088040    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d15fd3b-d5bb-46c4-9d9b-96f231782e98-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3d15fd3b-d5bb-46c4-9d9b-96f231782e98" (UID: "3d15fd3b-d5bb-46c4-9d9b-96f231782e98"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:01:17.189135 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:17.189101    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d15fd3b-d5bb-46c4-9d9b-96f231782e98-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:01:17.813391 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:17.813347    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97" event={"ID":"3d15fd3b-d5bb-46c4-9d9b-96f231782e98","Type":"ContainerDied","Data":"1b1d0efdc1d120cbf38efef49ccad86f22435ebbbd49b752691535d4fca97d91"}
Jul 17 10:01:17.813391 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:17.813386    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97"
Jul 17 10:01:17.813665 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:17.813433    2574 scope.go:117] "RemoveContainer" containerID="dcb07f769f3b4cacd6276316e2ea3a169e26c1d8c1fe3ad9308179c2eaaaa194"
Jul 17 10:01:17.843307 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:17.843271    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97"]
Jul 17 10:01:17.845445 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:17.845419    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-ddw97"]
Jul 17 10:01:18.898391 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:18.898360    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d15fd3b-d5bb-46c4-9d9b-96f231782e98" path="/var/lib/kubelet/pods/3d15fd3b-d5bb-46c4-9d9b-96f231782e98/volumes"
Jul 17 10:01:22.894327 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:22.894301    2574 scope.go:117] "RemoveContainer" containerID="c9e218f2618614b2c3ba0e7fd9b2895354b62d3534a4d121e37648c86fb6a399"
Jul 17 10:01:23.848764 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:23.848672    2574 generic.go:356] "Generic (PLEG): container finished" podID="c7914e85-46bd-49b9-9567-187f0dc2d576" containerID="a7a6f86295d4dfd258cc7273dae7cbbd92f74abe733fbbbc5013028a1e837daf" exitCode=6
Jul 17 10:01:23.848922 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:23.848752    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2crrz" event={"ID":"c7914e85-46bd-49b9-9567-187f0dc2d576","Type":"ContainerDied","Data":"a7a6f86295d4dfd258cc7273dae7cbbd92f74abe733fbbbc5013028a1e837daf"}
Jul 17 10:01:23.848922 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:23.848823    2574 scope.go:117] "RemoveContainer" containerID="c9e218f2618614b2c3ba0e7fd9b2895354b62d3534a4d121e37648c86fb6a399"
Jul 17 10:01:23.849180 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:23.849147    2574 scope.go:117] "RemoveContainer" containerID="a7a6f86295d4dfd258cc7273dae7cbbd92f74abe733fbbbc5013028a1e837daf"
Jul 17 10:01:23.849471 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:01:23.849445    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-2crrz_odh-ai-gateway-infra(c7914e85-46bd-49b9-9567-187f0dc2d576)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2crrz" podUID="c7914e85-46bd-49b9-9567-187f0dc2d576"
Jul 17 10:01:24.877673 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:24.877640    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2crrz"]
Jul 17 10:01:25.026990 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:25.026961    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2crrz"
Jul 17 10:01:25.168978 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:25.168949    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7914e85-46bd-49b9-9567-187f0dc2d576-maas-api-service-ca\") pod \"c7914e85-46bd-49b9-9567-187f0dc2d576\" (UID: \"c7914e85-46bd-49b9-9567-187f0dc2d576\") "
Jul 17 10:01:25.169361 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:25.169336    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7914e85-46bd-49b9-9567-187f0dc2d576-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c7914e85-46bd-49b9-9567-187f0dc2d576" (UID: "c7914e85-46bd-49b9-9567-187f0dc2d576"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:01:25.269905 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:25.269866    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7914e85-46bd-49b9-9567-187f0dc2d576-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:01:25.862925 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:25.862894    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2crrz"
Jul 17 10:01:25.863116 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:25.862923    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2crrz" event={"ID":"c7914e85-46bd-49b9-9567-187f0dc2d576","Type":"ContainerDied","Data":"6d45a630f0b99b86abbfae80ab63b98eaf30aa05e26dc3724024394a5614a581"}
Jul 17 10:01:25.863116 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:25.862990    2574 scope.go:117] "RemoveContainer" containerID="a7a6f86295d4dfd258cc7273dae7cbbd92f74abe733fbbbc5013028a1e837daf"
Jul 17 10:01:25.884484 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:25.884459    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2crrz"]
Jul 17 10:01:25.890363 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:25.890339    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2crrz"]
Jul 17 10:01:26.898124 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:26.898085    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7914e85-46bd-49b9-9567-187f0dc2d576" path="/var/lib/kubelet/pods/c7914e85-46bd-49b9-9567-187f0dc2d576/volumes"
Jul 17 10:01:37.865502 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:37.865473    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-m88fw"]
Jul 17 10:01:37.865966 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:37.865951    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d15fd3b-d5bb-46c4-9d9b-96f231782e98" containerName="revoke-keys"
Jul 17 10:01:37.866011 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:37.865967    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d15fd3b-d5bb-46c4-9d9b-96f231782e98" containerName="revoke-keys"
Jul 17 10:01:37.866011 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:37.865975    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7914e85-46bd-49b9-9567-187f0dc2d576" containerName="revoke-keys"
Jul 17 10:01:37.866011 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:37.865981    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7914e85-46bd-49b9-9567-187f0dc2d576" containerName="revoke-keys"
Jul 17 10:01:37.866011 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:37.865994    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7914e85-46bd-49b9-9567-187f0dc2d576" containerName="revoke-keys"
Jul 17 10:01:37.866011 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:37.866000    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7914e85-46bd-49b9-9567-187f0dc2d576" containerName="revoke-keys"
Jul 17 10:01:37.866011 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:37.866009    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d15fd3b-d5bb-46c4-9d9b-96f231782e98" containerName="revoke-keys"
Jul 17 10:01:37.866011 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:37.866013    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d15fd3b-d5bb-46c4-9d9b-96f231782e98" containerName="revoke-keys"
Jul 17 10:01:37.866225 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:37.866086    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d15fd3b-d5bb-46c4-9d9b-96f231782e98" containerName="revoke-keys"
Jul 17 10:01:37.866225 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:37.866094    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7914e85-46bd-49b9-9567-187f0dc2d576" containerName="revoke-keys"
Jul 17 10:01:37.866225 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:37.866103    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d15fd3b-d5bb-46c4-9d9b-96f231782e98" containerName="revoke-keys"
Jul 17 10:01:37.866225 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:37.866108    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d15fd3b-d5bb-46c4-9d9b-96f231782e98" containerName="revoke-keys"
Jul 17 10:01:37.866225 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:37.866113    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7914e85-46bd-49b9-9567-187f0dc2d576" containerName="revoke-keys"
Jul 17 10:01:37.866225 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:37.866117    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7914e85-46bd-49b9-9567-187f0dc2d576" containerName="revoke-keys"
Jul 17 10:01:37.870689 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:37.870672    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-m88fw"
Jul 17 10:01:37.874160 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:37.874137    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:01:37.879038 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:37.879013    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-m88fw"]
Jul 17 10:01:37.997873 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:37.997837    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9948c61-d978-4972-ae4d-6cf0b6304e95-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-m88fw\" (UID: \"e9948c61-d978-4972-ae4d-6cf0b6304e95\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-m88fw"
Jul 17 10:01:38.098751 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:38.098712    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9948c61-d978-4972-ae4d-6cf0b6304e95-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-m88fw\" (UID: \"e9948c61-d978-4972-ae4d-6cf0b6304e95\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-m88fw"
Jul 17 10:01:38.099398 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:38.099379    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9948c61-d978-4972-ae4d-6cf0b6304e95-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-m88fw\" (UID: \"e9948c61-d978-4972-ae4d-6cf0b6304e95\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-m88fw"
Jul 17 10:01:38.183948 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:38.183916    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-m88fw"
Jul 17 10:01:38.334452 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:38.334413    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-m88fw"]
Jul 17 10:01:38.335712 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:01:38.335680    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9948c61_d978_4972_ae4d_6cf0b6304e95.slice/crio-b06c7a33f3e1b763c0aefcdf2f92563bfa76a0e525d50ad13431398a3b157be2 WatchSource:0}: Error finding container b06c7a33f3e1b763c0aefcdf2f92563bfa76a0e525d50ad13431398a3b157be2: Status 404 returned error can't find the container with id b06c7a33f3e1b763c0aefcdf2f92563bfa76a0e525d50ad13431398a3b157be2
Jul 17 10:01:38.937208 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:38.937169    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-m88fw" event={"ID":"e9948c61-d978-4972-ae4d-6cf0b6304e95","Type":"ContainerStarted","Data":"b87a18f81228fbcfc416d144d466d5b214c5a7d2d7710d23bd376eadc8c05085"}
Jul 17 10:01:38.937602 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:38.937235    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-m88fw" event={"ID":"e9948c61-d978-4972-ae4d-6cf0b6304e95","Type":"ContainerStarted","Data":"b06c7a33f3e1b763c0aefcdf2f92563bfa76a0e525d50ad13431398a3b157be2"}
Jul 17 10:01:38.957364 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:38.957312    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-m88fw" podStartSLOduration=1.957293882 podStartE2EDuration="1.957293882s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:01:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:01:38.95490465 +0000 UTC m=+2096.537543024" watchObservedRunningTime="2026-07-17 10:01:38.957293882 +0000 UTC m=+2096.539932260"
Jul 17 10:01:39.947802 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:39.947763    2574 generic.go:356] "Generic (PLEG): container finished" podID="e9948c61-d978-4972-ae4d-6cf0b6304e95" containerID="b87a18f81228fbcfc416d144d466d5b214c5a7d2d7710d23bd376eadc8c05085" exitCode=6
Jul 17 10:01:39.948215 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:39.947849    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-m88fw" event={"ID":"e9948c61-d978-4972-ae4d-6cf0b6304e95","Type":"ContainerDied","Data":"b87a18f81228fbcfc416d144d466d5b214c5a7d2d7710d23bd376eadc8c05085"}
Jul 17 10:01:39.948215 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:39.948026    2574 scope.go:117] "RemoveContainer" containerID="b87a18f81228fbcfc416d144d466d5b214c5a7d2d7710d23bd376eadc8c05085"
Jul 17 10:01:40.954983 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:40.954949    2574 generic.go:356] "Generic (PLEG): container finished" podID="e9948c61-d978-4972-ae4d-6cf0b6304e95" containerID="f980ffbcd575cfcc78dbf4b71fd01ab18c36a56270970f9f2178debaffb7fda3" exitCode=6
Jul 17 10:01:40.955438 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:40.955033    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-m88fw" event={"ID":"e9948c61-d978-4972-ae4d-6cf0b6304e95","Type":"ContainerDied","Data":"f980ffbcd575cfcc78dbf4b71fd01ab18c36a56270970f9f2178debaffb7fda3"}
Jul 17 10:01:40.955438 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:40.955093    2574 scope.go:117] "RemoveContainer" containerID="b87a18f81228fbcfc416d144d466d5b214c5a7d2d7710d23bd376eadc8c05085"
Jul 17 10:01:40.955438 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:40.955202    2574 scope.go:117] "RemoveContainer" containerID="f980ffbcd575cfcc78dbf4b71fd01ab18c36a56270970f9f2178debaffb7fda3"
Jul 17 10:01:40.955575 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:01:40.955439    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-m88fw_odh-ai-gateway-infra(e9948c61-d978-4972-ae4d-6cf0b6304e95)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-m88fw" podUID="e9948c61-d978-4972-ae4d-6cf0b6304e95"
Jul 17 10:01:41.962313 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:41.962285    2574 scope.go:117] "RemoveContainer" containerID="f980ffbcd575cfcc78dbf4b71fd01ab18c36a56270970f9f2178debaffb7fda3"
Jul 17 10:01:41.962695 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:01:41.962488    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-m88fw_odh-ai-gateway-infra(e9948c61-d978-4972-ae4d-6cf0b6304e95)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-m88fw" podUID="e9948c61-d978-4972-ae4d-6cf0b6304e95"
Jul 17 10:01:43.036937 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:43.036830    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dh564_84db06ab-87e2-43b3-82d0-13f06ba86d06/ovn-acl-logging/0.log"
Jul 17 10:01:43.050933 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:43.041954    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dh564_84db06ab-87e2-43b3-82d0-13f06ba86d06/ovn-acl-logging/0.log"
Jul 17 10:01:55.757574 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:55.757534    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-kfv69"]
Jul 17 10:01:55.758067 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:55.757989    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d15fd3b-d5bb-46c4-9d9b-96f231782e98" containerName="revoke-keys"
Jul 17 10:01:55.758067 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:55.758001    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d15fd3b-d5bb-46c4-9d9b-96f231782e98" containerName="revoke-keys"
Jul 17 10:01:55.758067 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:55.758011    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7914e85-46bd-49b9-9567-187f0dc2d576" containerName="revoke-keys"
Jul 17 10:01:55.758067 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:55.758017    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7914e85-46bd-49b9-9567-187f0dc2d576" containerName="revoke-keys"
Jul 17 10:01:55.760995 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:55.760976    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-kfv69"
Jul 17 10:01:55.771936 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:55.771906    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-kfv69"]
Jul 17 10:01:55.868511 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:55.868467    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4263c957-28a2-47ff-8f7c-9646c085a4ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-kfv69\" (UID: \"4263c957-28a2-47ff-8f7c-9646c085a4ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-kfv69"
Jul 17 10:01:55.969932 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:55.969894    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4263c957-28a2-47ff-8f7c-9646c085a4ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-kfv69\" (UID: \"4263c957-28a2-47ff-8f7c-9646c085a4ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-kfv69"
Jul 17 10:01:55.970631 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:55.970611    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4263c957-28a2-47ff-8f7c-9646c085a4ba-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-kfv69\" (UID: \"4263c957-28a2-47ff-8f7c-9646c085a4ba\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-kfv69"
Jul 17 10:01:56.072128 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:56.072043    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-kfv69"
Jul 17 10:01:56.216536 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:56.216504    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-kfv69"]
Jul 17 10:01:56.219931 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:01:56.219899    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4263c957_28a2_47ff_8f7c_9646c085a4ba.slice/crio-a5e44ce52aaa84157614d5ad96af0090b1e28cc9cae8a792941079ff49b18d15 WatchSource:0}: Error finding container a5e44ce52aaa84157614d5ad96af0090b1e28cc9cae8a792941079ff49b18d15: Status 404 returned error can't find the container with id a5e44ce52aaa84157614d5ad96af0090b1e28cc9cae8a792941079ff49b18d15
Jul 17 10:01:56.707001 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:56.706899    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-5n66c"]
Jul 17 10:01:56.711199 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:56.711170    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-5n66c"
Jul 17 10:01:56.729260 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:56.727980    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-5n66c"]
Jul 17 10:01:56.779222 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:56.779192    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/928cb59b-8a75-4867-8ef3-2c7ef7fb8539-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-5n66c\" (UID: \"928cb59b-8a75-4867-8ef3-2c7ef7fb8539\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-5n66c"
Jul 17 10:01:56.879998 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:56.879957    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/928cb59b-8a75-4867-8ef3-2c7ef7fb8539-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-5n66c\" (UID: \"928cb59b-8a75-4867-8ef3-2c7ef7fb8539\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-5n66c"
Jul 17 10:01:56.880587 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:56.880566    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/928cb59b-8a75-4867-8ef3-2c7ef7fb8539-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-5n66c\" (UID: \"928cb59b-8a75-4867-8ef3-2c7ef7fb8539\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-5n66c"
Jul 17 10:01:56.898146 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:56.898118    2574 scope.go:117] "RemoveContainer" containerID="f980ffbcd575cfcc78dbf4b71fd01ab18c36a56270970f9f2178debaffb7fda3"
Jul 17 10:01:57.034787 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:57.034757    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-5n66c"
Jul 17 10:01:57.049992 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:57.049963    2574 generic.go:356] "Generic (PLEG): container finished" podID="4263c957-28a2-47ff-8f7c-9646c085a4ba" containerID="bb4657bc73ac4b647aff39aaa799ef647e23654c2d1e210b1177c1beff8eacdc" exitCode=7
Jul 17 10:01:57.050131 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:57.050053    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-kfv69" event={"ID":"4263c957-28a2-47ff-8f7c-9646c085a4ba","Type":"ContainerDied","Data":"bb4657bc73ac4b647aff39aaa799ef647e23654c2d1e210b1177c1beff8eacdc"}
Jul 17 10:01:57.050131 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:57.050117    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-kfv69" event={"ID":"4263c957-28a2-47ff-8f7c-9646c085a4ba","Type":"ContainerStarted","Data":"a5e44ce52aaa84157614d5ad96af0090b1e28cc9cae8a792941079ff49b18d15"}
Jul 17 10:01:57.050362 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:57.050348    2574 scope.go:117] "RemoveContainer" containerID="bb4657bc73ac4b647aff39aaa799ef647e23654c2d1e210b1177c1beff8eacdc"
Jul 17 10:01:57.189572 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:57.189546    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-5n66c"]
Jul 17 10:01:57.191500 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:01:57.191474    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod928cb59b_8a75_4867_8ef3_2c7ef7fb8539.slice/crio-490177a193e754f53dfd02d01e7bd29824411aa15164bd85714a5e6c62d6b80e WatchSource:0}: Error finding container 490177a193e754f53dfd02d01e7bd29824411aa15164bd85714a5e6c62d6b80e: Status 404 returned error can't find the container with id 490177a193e754f53dfd02d01e7bd29824411aa15164bd85714a5e6c62d6b80e
Jul 17 10:01:58.056847 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:58.056809    2574 generic.go:356] "Generic (PLEG): container finished" podID="928cb59b-8a75-4867-8ef3-2c7ef7fb8539" containerID="fdeee74783541f3798530667e09f71f53c94f4ca82efcb1c9c5dd524bb43b3dd" exitCode=6
Jul 17 10:01:58.057378 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:58.056914    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-5n66c" event={"ID":"928cb59b-8a75-4867-8ef3-2c7ef7fb8539","Type":"ContainerDied","Data":"fdeee74783541f3798530667e09f71f53c94f4ca82efcb1c9c5dd524bb43b3dd"}
Jul 17 10:01:58.057378 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:58.056973    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-5n66c" event={"ID":"928cb59b-8a75-4867-8ef3-2c7ef7fb8539","Type":"ContainerStarted","Data":"490177a193e754f53dfd02d01e7bd29824411aa15164bd85714a5e6c62d6b80e"}
Jul 17 10:01:58.057378 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:58.057113    2574 scope.go:117] "RemoveContainer" containerID="fdeee74783541f3798530667e09f71f53c94f4ca82efcb1c9c5dd524bb43b3dd"
Jul 17 10:01:58.058996 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:58.058973    2574 generic.go:356] "Generic (PLEG): container finished" podID="e9948c61-d978-4972-ae4d-6cf0b6304e95" containerID="13b9293a5879b04f7ec05d64de618a4345ea6f77daceb46644499f5ebce77a15" exitCode=6
Jul 17 10:01:58.059146 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:58.059050    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-m88fw" event={"ID":"e9948c61-d978-4972-ae4d-6cf0b6304e95","Type":"ContainerDied","Data":"13b9293a5879b04f7ec05d64de618a4345ea6f77daceb46644499f5ebce77a15"}
Jul 17 10:01:58.059146 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:58.059111    2574 scope.go:117] "RemoveContainer" containerID="f980ffbcd575cfcc78dbf4b71fd01ab18c36a56270970f9f2178debaffb7fda3"
Jul 17 10:01:58.059437 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:58.059419    2574 scope.go:117] "RemoveContainer" containerID="13b9293a5879b04f7ec05d64de618a4345ea6f77daceb46644499f5ebce77a15"
Jul 17 10:01:58.059692 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:01:58.059653    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-m88fw_odh-ai-gateway-infra(e9948c61-d978-4972-ae4d-6cf0b6304e95)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-m88fw" podUID="e9948c61-d978-4972-ae4d-6cf0b6304e95"
Jul 17 10:01:58.061251 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:58.061207    2574 generic.go:356] "Generic (PLEG): container finished" podID="4263c957-28a2-47ff-8f7c-9646c085a4ba" containerID="b39779d02d4b525e44c72ae43512a7e62e4c5f3c70b9e9c91a0ab4ce9b24a10a" exitCode=7
Jul 17 10:01:58.061314 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:58.061259    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-kfv69" event={"ID":"4263c957-28a2-47ff-8f7c-9646c085a4ba","Type":"ContainerDied","Data":"b39779d02d4b525e44c72ae43512a7e62e4c5f3c70b9e9c91a0ab4ce9b24a10a"}
Jul 17 10:01:58.061491 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:58.061476    2574 scope.go:117] "RemoveContainer" containerID="b39779d02d4b525e44c72ae43512a7e62e4c5f3c70b9e9c91a0ab4ce9b24a10a"
Jul 17 10:01:58.061667 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:01:58.061651    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-kfv69_odh-ai-gateway-infra(4263c957-28a2-47ff-8f7c-9646c085a4ba)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-kfv69" podUID="4263c957-28a2-47ff-8f7c-9646c085a4ba"
Jul 17 10:01:58.074178 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:58.074146    2574 scope.go:117] "RemoveContainer" containerID="bb4657bc73ac4b647aff39aaa799ef647e23654c2d1e210b1177c1beff8eacdc"
Jul 17 10:01:59.068623 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:59.068581    2574 generic.go:356] "Generic (PLEG): container finished" podID="928cb59b-8a75-4867-8ef3-2c7ef7fb8539" containerID="c871c491f3bc38e20ec6a04d8b2bbaa04121999ed760a75172e81dc6a4d35e8c" exitCode=6
Jul 17 10:01:59.069108 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:59.068669    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-5n66c" event={"ID":"928cb59b-8a75-4867-8ef3-2c7ef7fb8539","Type":"ContainerDied","Data":"c871c491f3bc38e20ec6a04d8b2bbaa04121999ed760a75172e81dc6a4d35e8c"}
Jul 17 10:01:59.069108 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:59.068734    2574 scope.go:117] "RemoveContainer" containerID="fdeee74783541f3798530667e09f71f53c94f4ca82efcb1c9c5dd524bb43b3dd"
Jul 17 10:01:59.069108 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:59.069040    2574 scope.go:117] "RemoveContainer" containerID="c871c491f3bc38e20ec6a04d8b2bbaa04121999ed760a75172e81dc6a4d35e8c"
Jul 17 10:01:59.069333 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:01:59.069311    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-5n66c_odh-ai-gateway-infra(928cb59b-8a75-4867-8ef3-2c7ef7fb8539)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-5n66c" podUID="928cb59b-8a75-4867-8ef3-2c7ef7fb8539"
Jul 17 10:01:59.072192 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:59.072170    2574 scope.go:117] "RemoveContainer" containerID="b39779d02d4b525e44c72ae43512a7e62e4c5f3c70b9e9c91a0ab4ce9b24a10a"
Jul 17 10:01:59.072397 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:01:59.072378    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-kfv69_odh-ai-gateway-infra(4263c957-28a2-47ff-8f7c-9646c085a4ba)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-kfv69" podUID="4263c957-28a2-47ff-8f7c-9646c085a4ba"
Jul 17 10:01:59.102067 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:59.102035    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-m88fw"]
Jul 17 10:01:59.248716 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:59.248693    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-m88fw"
Jul 17 10:01:59.302035 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:59.302001    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9948c61-d978-4972-ae4d-6cf0b6304e95-maas-api-service-ca\") pod \"e9948c61-d978-4972-ae4d-6cf0b6304e95\" (UID: \"e9948c61-d978-4972-ae4d-6cf0b6304e95\") "
Jul 17 10:01:59.302361 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:59.302337    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9948c61-d978-4972-ae4d-6cf0b6304e95-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e9948c61-d978-4972-ae4d-6cf0b6304e95" (UID: "e9948c61-d978-4972-ae4d-6cf0b6304e95"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:01:59.402766 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:01:59.402679    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9948c61-d978-4972-ae4d-6cf0b6304e95-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:02:00.079425 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:00.079386    2574 scope.go:117] "RemoveContainer" containerID="c871c491f3bc38e20ec6a04d8b2bbaa04121999ed760a75172e81dc6a4d35e8c"
Jul 17 10:02:00.079842 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:02:00.079641    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-5n66c_odh-ai-gateway-infra(928cb59b-8a75-4867-8ef3-2c7ef7fb8539)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-5n66c" podUID="928cb59b-8a75-4867-8ef3-2c7ef7fb8539"
Jul 17 10:02:00.080398 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:00.080381    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-m88fw"
Jul 17 10:02:00.080482 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:00.080381    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-m88fw" event={"ID":"e9948c61-d978-4972-ae4d-6cf0b6304e95","Type":"ContainerDied","Data":"b06c7a33f3e1b763c0aefcdf2f92563bfa76a0e525d50ad13431398a3b157be2"}
Jul 17 10:02:00.080543 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:00.080529    2574 scope.go:117] "RemoveContainer" containerID="13b9293a5879b04f7ec05d64de618a4345ea6f77daceb46644499f5ebce77a15"
Jul 17 10:02:00.111517 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:00.111477    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-m88fw"]
Jul 17 10:02:00.120620 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:00.120586    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-m88fw"]
Jul 17 10:02:00.898974 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:00.898938    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9948c61-d978-4972-ae4d-6cf0b6304e95" path="/var/lib/kubelet/pods/e9948c61-d978-4972-ae4d-6cf0b6304e95/volumes"
Jul 17 10:02:12.894815 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:12.894780    2574 scope.go:117] "RemoveContainer" containerID="b39779d02d4b525e44c72ae43512a7e62e4c5f3c70b9e9c91a0ab4ce9b24a10a"
Jul 17 10:02:13.891622 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:13.891536    2574 scope.go:117] "RemoveContainer" containerID="c871c491f3bc38e20ec6a04d8b2bbaa04121999ed760a75172e81dc6a4d35e8c"
Jul 17 10:02:14.156902 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:14.156807    2574 generic.go:356] "Generic (PLEG): container finished" podID="4263c957-28a2-47ff-8f7c-9646c085a4ba" containerID="16829e1d3eaeb9a301a1e4ed7d4411a85ee2ec3ac994d1699ef117a82acfc9e2" exitCode=7
Jul 17 10:02:14.156902 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:14.156858    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-kfv69" event={"ID":"4263c957-28a2-47ff-8f7c-9646c085a4ba","Type":"ContainerDied","Data":"16829e1d3eaeb9a301a1e4ed7d4411a85ee2ec3ac994d1699ef117a82acfc9e2"}
Jul 17 10:02:14.157397 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:14.156955    2574 scope.go:117] "RemoveContainer" containerID="b39779d02d4b525e44c72ae43512a7e62e4c5f3c70b9e9c91a0ab4ce9b24a10a"
Jul 17 10:02:14.157397 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:14.157232    2574 scope.go:117] "RemoveContainer" containerID="16829e1d3eaeb9a301a1e4ed7d4411a85ee2ec3ac994d1699ef117a82acfc9e2"
Jul 17 10:02:14.157543 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:02:14.157521    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-kfv69_odh-ai-gateway-infra(4263c957-28a2-47ff-8f7c-9646c085a4ba)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-kfv69" podUID="4263c957-28a2-47ff-8f7c-9646c085a4ba"
Jul 17 10:02:15.165747 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:15.165712    2574 generic.go:356] "Generic (PLEG): container finished" podID="928cb59b-8a75-4867-8ef3-2c7ef7fb8539" containerID="3e33768cf32468b23a53e6f93d0dec69e7c4af3c9282938b88811576128cbfea" exitCode=6
Jul 17 10:02:15.166217 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:15.165794    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-5n66c" event={"ID":"928cb59b-8a75-4867-8ef3-2c7ef7fb8539","Type":"ContainerDied","Data":"3e33768cf32468b23a53e6f93d0dec69e7c4af3c9282938b88811576128cbfea"}
Jul 17 10:02:15.166217 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:15.165861    2574 scope.go:117] "RemoveContainer" containerID="c871c491f3bc38e20ec6a04d8b2bbaa04121999ed760a75172e81dc6a4d35e8c"
Jul 17 10:02:15.166389 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:15.166262    2574 scope.go:117] "RemoveContainer" containerID="3e33768cf32468b23a53e6f93d0dec69e7c4af3c9282938b88811576128cbfea"
Jul 17 10:02:15.166536 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:02:15.166516    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-5n66c_odh-ai-gateway-infra(928cb59b-8a75-4867-8ef3-2c7ef7fb8539)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-5n66c" podUID="928cb59b-8a75-4867-8ef3-2c7ef7fb8539"
Jul 17 10:02:15.192853 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:15.192822    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-kfv69"]
Jul 17 10:02:15.336175 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:15.336151    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-kfv69"
Jul 17 10:02:15.465899 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:15.465811    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4263c957-28a2-47ff-8f7c-9646c085a4ba-maas-api-service-ca\") pod \"4263c957-28a2-47ff-8f7c-9646c085a4ba\" (UID: \"4263c957-28a2-47ff-8f7c-9646c085a4ba\") "
Jul 17 10:02:15.466167 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:15.466145    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4263c957-28a2-47ff-8f7c-9646c085a4ba-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4263c957-28a2-47ff-8f7c-9646c085a4ba" (UID: "4263c957-28a2-47ff-8f7c-9646c085a4ba"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:02:15.566678 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:15.566643    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4263c957-28a2-47ff-8f7c-9646c085a4ba-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:02:16.174686 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:16.174660    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-kfv69"
Jul 17 10:02:16.174686 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:16.174664    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-kfv69" event={"ID":"4263c957-28a2-47ff-8f7c-9646c085a4ba","Type":"ContainerDied","Data":"a5e44ce52aaa84157614d5ad96af0090b1e28cc9cae8a792941079ff49b18d15"}
Jul 17 10:02:16.175168 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:16.174721    2574 scope.go:117] "RemoveContainer" containerID="16829e1d3eaeb9a301a1e4ed7d4411a85ee2ec3ac994d1699ef117a82acfc9e2"
Jul 17 10:02:16.193298 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:16.193273    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-5n66c"]
Jul 17 10:02:16.198762 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:16.198738    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-kfv69"]
Jul 17 10:02:16.203109 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:16.203086    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-kfv69"]
Jul 17 10:02:16.343653 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:16.343629    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-5n66c"
Jul 17 10:02:16.474155 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:16.474072    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/928cb59b-8a75-4867-8ef3-2c7ef7fb8539-maas-api-service-ca\") pod \"928cb59b-8a75-4867-8ef3-2c7ef7fb8539\" (UID: \"928cb59b-8a75-4867-8ef3-2c7ef7fb8539\") "
Jul 17 10:02:16.474462 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:16.474440    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/928cb59b-8a75-4867-8ef3-2c7ef7fb8539-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "928cb59b-8a75-4867-8ef3-2c7ef7fb8539" (UID: "928cb59b-8a75-4867-8ef3-2c7ef7fb8539"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:02:16.575459 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:16.575427    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/928cb59b-8a75-4867-8ef3-2c7ef7fb8539-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:02:16.897846 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:16.897757    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4263c957-28a2-47ff-8f7c-9646c085a4ba" path="/var/lib/kubelet/pods/4263c957-28a2-47ff-8f7c-9646c085a4ba/volumes"
Jul 17 10:02:17.182103 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:17.182068    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-5n66c" event={"ID":"928cb59b-8a75-4867-8ef3-2c7ef7fb8539","Type":"ContainerDied","Data":"490177a193e754f53dfd02d01e7bd29824411aa15164bd85714a5e6c62d6b80e"}
Jul 17 10:02:17.182592 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:17.182107    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-5n66c"
Jul 17 10:02:17.182592 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:17.182139    2574 scope.go:117] "RemoveContainer" containerID="3e33768cf32468b23a53e6f93d0dec69e7c4af3c9282938b88811576128cbfea"
Jul 17 10:02:17.200459 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:17.200431    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-5n66c"]
Jul 17 10:02:17.204748 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:17.204727    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-5n66c"]
Jul 17 10:02:18.898217 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:18.898181    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="928cb59b-8a75-4867-8ef3-2c7ef7fb8539" path="/var/lib/kubelet/pods/928cb59b-8a75-4867-8ef3-2c7ef7fb8539/volumes"
Jul 17 10:02:37.904812 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:37.904729    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-pchnr"]
Jul 17 10:02:37.905228 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:37.905158    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4263c957-28a2-47ff-8f7c-9646c085a4ba" containerName="revoke-keys"
Jul 17 10:02:37.905228 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:37.905168    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4263c957-28a2-47ff-8f7c-9646c085a4ba" containerName="revoke-keys"
Jul 17 10:02:37.905228 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:37.905175    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="928cb59b-8a75-4867-8ef3-2c7ef7fb8539" containerName="revoke-keys"
Jul 17 10:02:37.905228 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:37.905181    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="928cb59b-8a75-4867-8ef3-2c7ef7fb8539" containerName="revoke-keys"
Jul 17 10:02:37.905228 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:37.905188    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4263c957-28a2-47ff-8f7c-9646c085a4ba" containerName="revoke-keys"
Jul 17 10:02:37.905228 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:37.905194    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4263c957-28a2-47ff-8f7c-9646c085a4ba" containerName="revoke-keys"
Jul 17 10:02:37.905228 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:37.905204    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9948c61-d978-4972-ae4d-6cf0b6304e95" containerName="revoke-keys"
Jul 17 10:02:37.905228 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:37.905209    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9948c61-d978-4972-ae4d-6cf0b6304e95" containerName="revoke-keys"
Jul 17 10:02:37.905228 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:37.905222    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9948c61-d978-4972-ae4d-6cf0b6304e95" containerName="revoke-keys"
Jul 17 10:02:37.905228 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:37.905227    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9948c61-d978-4972-ae4d-6cf0b6304e95" containerName="revoke-keys"
Jul 17 10:02:37.905228 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:37.905249    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="928cb59b-8a75-4867-8ef3-2c7ef7fb8539" containerName="revoke-keys"
Jul 17 10:02:37.905228 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:37.905255    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="928cb59b-8a75-4867-8ef3-2c7ef7fb8539" containerName="revoke-keys"
Jul 17 10:02:37.905653 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:37.905335    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4263c957-28a2-47ff-8f7c-9646c085a4ba" containerName="revoke-keys"
Jul 17 10:02:37.905653 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:37.905343    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="928cb59b-8a75-4867-8ef3-2c7ef7fb8539" containerName="revoke-keys"
Jul 17 10:02:37.905653 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:37.905349    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4263c957-28a2-47ff-8f7c-9646c085a4ba" containerName="revoke-keys"
Jul 17 10:02:37.905653 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:37.905355    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="928cb59b-8a75-4867-8ef3-2c7ef7fb8539" containerName="revoke-keys"
Jul 17 10:02:37.905653 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:37.905361    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9948c61-d978-4972-ae4d-6cf0b6304e95" containerName="revoke-keys"
Jul 17 10:02:37.905653 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:37.905368    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9948c61-d978-4972-ae4d-6cf0b6304e95" containerName="revoke-keys"
Jul 17 10:02:37.910278 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:37.910263    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-pchnr"
Jul 17 10:02:37.913281 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:37.913221    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:02:37.915104 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:37.915082    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-pchnr"]
Jul 17 10:02:37.976122 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:37.976073    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d2f216e-3777-463c-8197-66b31bdfa794-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-pchnr\" (UID: \"8d2f216e-3777-463c-8197-66b31bdfa794\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-pchnr"
Jul 17 10:02:38.076934 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:38.076894    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d2f216e-3777-463c-8197-66b31bdfa794-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-pchnr\" (UID: \"8d2f216e-3777-463c-8197-66b31bdfa794\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-pchnr"
Jul 17 10:02:38.077570 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:38.077548    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d2f216e-3777-463c-8197-66b31bdfa794-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-pchnr\" (UID: \"8d2f216e-3777-463c-8197-66b31bdfa794\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-pchnr"
Jul 17 10:02:38.222154 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:38.222112    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-pchnr"
Jul 17 10:02:38.357648 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:38.357620    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-pchnr"]
Jul 17 10:02:38.358972 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:02:38.358945    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d2f216e_3777_463c_8197_66b31bdfa794.slice/crio-84252c921021729c14a81c8d4136fa76d96fa694c455d4c76e87e7dd13000b91 WatchSource:0}: Error finding container 84252c921021729c14a81c8d4136fa76d96fa694c455d4c76e87e7dd13000b91: Status 404 returned error can't find the container with id 84252c921021729c14a81c8d4136fa76d96fa694c455d4c76e87e7dd13000b91
Jul 17 10:02:39.309376 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:39.309268    2574 generic.go:356] "Generic (PLEG): container finished" podID="8d2f216e-3777-463c-8197-66b31bdfa794" containerID="16bc1c588dfe5b2a44ceb404f1180f708399c84f8dcd45a2dc179841ad416478" exitCode=6
Jul 17 10:02:39.309376 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:39.309351    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-pchnr" event={"ID":"8d2f216e-3777-463c-8197-66b31bdfa794","Type":"ContainerDied","Data":"16bc1c588dfe5b2a44ceb404f1180f708399c84f8dcd45a2dc179841ad416478"}
Jul 17 10:02:39.309806 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:39.309417    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-pchnr" event={"ID":"8d2f216e-3777-463c-8197-66b31bdfa794","Type":"ContainerStarted","Data":"84252c921021729c14a81c8d4136fa76d96fa694c455d4c76e87e7dd13000b91"}
Jul 17 10:02:39.309806 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:39.309573    2574 scope.go:117] "RemoveContainer" containerID="16bc1c588dfe5b2a44ceb404f1180f708399c84f8dcd45a2dc179841ad416478"
Jul 17 10:02:40.317007 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:40.316976    2574 generic.go:356] "Generic (PLEG): container finished" podID="8d2f216e-3777-463c-8197-66b31bdfa794" containerID="a7dc6781016c16a8579722f145954b2b68c878c64c988535f0f4d7c57afc0ce8" exitCode=6
Jul 17 10:02:40.317419 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:40.317048    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-pchnr" event={"ID":"8d2f216e-3777-463c-8197-66b31bdfa794","Type":"ContainerDied","Data":"a7dc6781016c16a8579722f145954b2b68c878c64c988535f0f4d7c57afc0ce8"}
Jul 17 10:02:40.317419 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:40.317096    2574 scope.go:117] "RemoveContainer" containerID="16bc1c588dfe5b2a44ceb404f1180f708399c84f8dcd45a2dc179841ad416478"
Jul 17 10:02:40.317419 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:40.317320    2574 scope.go:117] "RemoveContainer" containerID="a7dc6781016c16a8579722f145954b2b68c878c64c988535f0f4d7c57afc0ce8"
Jul 17 10:02:40.317600 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:02:40.317576    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-pchnr_odh-ai-gateway-infra(8d2f216e-3777-463c-8197-66b31bdfa794)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-pchnr" podUID="8d2f216e-3777-463c-8197-66b31bdfa794"
Jul 17 10:02:41.324515 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:41.324490    2574 scope.go:117] "RemoveContainer" containerID="a7dc6781016c16a8579722f145954b2b68c878c64c988535f0f4d7c57afc0ce8"
Jul 17 10:02:41.324992 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:02:41.324749    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-pchnr_odh-ai-gateway-infra(8d2f216e-3777-463c-8197-66b31bdfa794)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-pchnr" podUID="8d2f216e-3777-463c-8197-66b31bdfa794"
Jul 17 10:02:52.894598 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:52.894572    2574 scope.go:117] "RemoveContainer" containerID="a7dc6781016c16a8579722f145954b2b68c878c64c988535f0f4d7c57afc0ce8"
Jul 17 10:02:53.392567 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:53.392530    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-pchnr" event={"ID":"8d2f216e-3777-463c-8197-66b31bdfa794","Type":"ContainerStarted","Data":"e62e7636a36cb8f13f1b8ff11fff0456df3bb1053eb8c6490152f4c93ce1b016"}
Jul 17 10:02:53.409118 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:53.409066    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-pchnr" podStartSLOduration=16.409048998 podStartE2EDuration="16.409048998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:02:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:02:53.408290852 +0000 UTC m=+2170.990929228" watchObservedRunningTime="2026-07-17 10:02:53.409048998 +0000 UTC m=+2170.991687373"
Jul 17 10:02:54.399609 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:54.399577    2574 generic.go:356] "Generic (PLEG): container finished" podID="8d2f216e-3777-463c-8197-66b31bdfa794" containerID="e62e7636a36cb8f13f1b8ff11fff0456df3bb1053eb8c6490152f4c93ce1b016" exitCode=6
Jul 17 10:02:54.400124 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:54.399619    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-pchnr" event={"ID":"8d2f216e-3777-463c-8197-66b31bdfa794","Type":"ContainerDied","Data":"e62e7636a36cb8f13f1b8ff11fff0456df3bb1053eb8c6490152f4c93ce1b016"}
Jul 17 10:02:54.400124 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:54.399680    2574 scope.go:117] "RemoveContainer" containerID="a7dc6781016c16a8579722f145954b2b68c878c64c988535f0f4d7c57afc0ce8"
Jul 17 10:02:54.400124 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:54.399972    2574 scope.go:117] "RemoveContainer" containerID="e62e7636a36cb8f13f1b8ff11fff0456df3bb1053eb8c6490152f4c93ce1b016"
Jul 17 10:02:54.400324 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:02:54.400193    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-pchnr_odh-ai-gateway-infra(8d2f216e-3777-463c-8197-66b31bdfa794)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-pchnr" podUID="8d2f216e-3777-463c-8197-66b31bdfa794"
Jul 17 10:02:54.424886 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:54.424856    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-pchnr"]
Jul 17 10:02:55.551616 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:55.551590    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-pchnr"
Jul 17 10:02:55.643221 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:55.643187    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d2f216e-3777-463c-8197-66b31bdfa794-maas-api-service-ca\") pod \"8d2f216e-3777-463c-8197-66b31bdfa794\" (UID: \"8d2f216e-3777-463c-8197-66b31bdfa794\") "
Jul 17 10:02:55.643602 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:55.643582    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d2f216e-3777-463c-8197-66b31bdfa794-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8d2f216e-3777-463c-8197-66b31bdfa794" (UID: "8d2f216e-3777-463c-8197-66b31bdfa794"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:02:55.744109 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:55.744080    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8d2f216e-3777-463c-8197-66b31bdfa794-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:02:55.804981 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:55.804946    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-r7xhf"]
Jul 17 10:02:55.805699 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:55.805680    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d2f216e-3777-463c-8197-66b31bdfa794" containerName="revoke-keys"
Jul 17 10:02:55.805769 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:55.805702    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d2f216e-3777-463c-8197-66b31bdfa794" containerName="revoke-keys"
Jul 17 10:02:55.805769 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:55.805716    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d2f216e-3777-463c-8197-66b31bdfa794" containerName="revoke-keys"
Jul 17 10:02:55.805769 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:55.805724    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d2f216e-3777-463c-8197-66b31bdfa794" containerName="revoke-keys"
Jul 17 10:02:55.805769 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:55.805742    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4263c957-28a2-47ff-8f7c-9646c085a4ba" containerName="revoke-keys"
Jul 17 10:02:55.805769 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:55.805751    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4263c957-28a2-47ff-8f7c-9646c085a4ba" containerName="revoke-keys"
Jul 17 10:02:55.805928 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:55.805780    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9948c61-d978-4972-ae4d-6cf0b6304e95" containerName="revoke-keys"
Jul 17 10:02:55.805928 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:55.805789    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9948c61-d978-4972-ae4d-6cf0b6304e95" containerName="revoke-keys"
Jul 17 10:02:55.805928 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:55.805803    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="928cb59b-8a75-4867-8ef3-2c7ef7fb8539" containerName="revoke-keys"
Jul 17 10:02:55.805928 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:55.805811    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="928cb59b-8a75-4867-8ef3-2c7ef7fb8539" containerName="revoke-keys"
Jul 17 10:02:55.805928 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:55.805916    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4263c957-28a2-47ff-8f7c-9646c085a4ba" containerName="revoke-keys"
Jul 17 10:02:55.806088 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:55.805929    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="928cb59b-8a75-4867-8ef3-2c7ef7fb8539" containerName="revoke-keys"
Jul 17 10:02:55.806088 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:55.805940    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9948c61-d978-4972-ae4d-6cf0b6304e95" containerName="revoke-keys"
Jul 17 10:02:55.806088 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:55.805950    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d2f216e-3777-463c-8197-66b31bdfa794" containerName="revoke-keys"
Jul 17 10:02:55.806088 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:55.805960    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d2f216e-3777-463c-8197-66b31bdfa794" containerName="revoke-keys"
Jul 17 10:02:55.809332 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:55.809311    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-r7xhf"
Jul 17 10:02:55.814544 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:55.814515    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-r7xhf"]
Jul 17 10:02:55.946382 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:55.946339    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c90a0362-eb84-45f1-aa4b-ac3a37ec46f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-r7xhf\" (UID: \"c90a0362-eb84-45f1-aa4b-ac3a37ec46f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-r7xhf"
Jul 17 10:02:56.047928 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:56.047827    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c90a0362-eb84-45f1-aa4b-ac3a37ec46f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-r7xhf\" (UID: \"c90a0362-eb84-45f1-aa4b-ac3a37ec46f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-r7xhf"
Jul 17 10:02:56.048691 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:56.048668    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c90a0362-eb84-45f1-aa4b-ac3a37ec46f7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-r7xhf\" (UID: \"c90a0362-eb84-45f1-aa4b-ac3a37ec46f7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-r7xhf"
Jul 17 10:02:56.136788 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:56.136754    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-r7xhf"
Jul 17 10:02:56.272147 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:56.272118    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-r7xhf"]
Jul 17 10:02:56.274713 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:02:56.274670    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc90a0362_eb84_45f1_aa4b_ac3a37ec46f7.slice/crio-0730d364d453d64d57259743ece54e11da6ebe7c1814c6e86f1b05b8825a3e89 WatchSource:0}: Error finding container 0730d364d453d64d57259743ece54e11da6ebe7c1814c6e86f1b05b8825a3e89: Status 404 returned error can't find the container with id 0730d364d453d64d57259743ece54e11da6ebe7c1814c6e86f1b05b8825a3e89
Jul 17 10:02:56.413662 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:56.413637    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-pchnr"
Jul 17 10:02:56.413855 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:56.413630    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-pchnr" event={"ID":"8d2f216e-3777-463c-8197-66b31bdfa794","Type":"ContainerDied","Data":"84252c921021729c14a81c8d4136fa76d96fa694c455d4c76e87e7dd13000b91"}
Jul 17 10:02:56.413855 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:56.413781    2574 scope.go:117] "RemoveContainer" containerID="e62e7636a36cb8f13f1b8ff11fff0456df3bb1053eb8c6490152f4c93ce1b016"
Jul 17 10:02:56.415144 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:56.414987    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-r7xhf" event={"ID":"c90a0362-eb84-45f1-aa4b-ac3a37ec46f7","Type":"ContainerStarted","Data":"0730d364d453d64d57259743ece54e11da6ebe7c1814c6e86f1b05b8825a3e89"}
Jul 17 10:02:56.438011 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:56.437990    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-pchnr"]
Jul 17 10:02:56.440649 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:56.440621    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-pchnr"]
Jul 17 10:02:56.751159 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:56.751123    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mpgtb"]
Jul 17 10:02:56.751593 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:56.751579    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d2f216e-3777-463c-8197-66b31bdfa794" containerName="revoke-keys"
Jul 17 10:02:56.751593 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:56.751594    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d2f216e-3777-463c-8197-66b31bdfa794" containerName="revoke-keys"
Jul 17 10:02:56.751682 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:56.751655    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d2f216e-3777-463c-8197-66b31bdfa794" containerName="revoke-keys"
Jul 17 10:02:56.754730 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:56.754710    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mpgtb"
Jul 17 10:02:56.762254 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:56.762201    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mpgtb"]
Jul 17 10:02:56.855627 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:56.855587    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33352f89-e1a0-4192-96c7-279d9e72c734-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-mpgtb\" (UID: \"33352f89-e1a0-4192-96c7-279d9e72c734\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mpgtb"
Jul 17 10:02:56.898809 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:56.898731    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d2f216e-3777-463c-8197-66b31bdfa794" path="/var/lib/kubelet/pods/8d2f216e-3777-463c-8197-66b31bdfa794/volumes"
Jul 17 10:02:56.956700 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:56.956659    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33352f89-e1a0-4192-96c7-279d9e72c734-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-mpgtb\" (UID: \"33352f89-e1a0-4192-96c7-279d9e72c734\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mpgtb"
Jul 17 10:02:56.957293 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:56.957273    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33352f89-e1a0-4192-96c7-279d9e72c734-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-mpgtb\" (UID: \"33352f89-e1a0-4192-96c7-279d9e72c734\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mpgtb"
Jul 17 10:02:57.067525 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:57.067490    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mpgtb"
Jul 17 10:02:57.207715 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:57.207690    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mpgtb"]
Jul 17 10:02:57.211280 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:02:57.211232    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33352f89_e1a0_4192_96c7_279d9e72c734.slice/crio-fa3e553fe0bf4afe1c368ec84ba9f4ac48b96742efd20a716a0dd69ba0fc9583 WatchSource:0}: Error finding container fa3e553fe0bf4afe1c368ec84ba9f4ac48b96742efd20a716a0dd69ba0fc9583: Status 404 returned error can't find the container with id fa3e553fe0bf4afe1c368ec84ba9f4ac48b96742efd20a716a0dd69ba0fc9583
Jul 17 10:02:57.425017 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:57.424977    2574 generic.go:356] "Generic (PLEG): container finished" podID="c90a0362-eb84-45f1-aa4b-ac3a37ec46f7" containerID="43c22adc281abd1c29cdc0edc94209b6723d53d963ca71860f5409a6580026de" exitCode=7
Jul 17 10:02:57.425174 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:57.425059    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-r7xhf" event={"ID":"c90a0362-eb84-45f1-aa4b-ac3a37ec46f7","Type":"ContainerDied","Data":"43c22adc281abd1c29cdc0edc94209b6723d53d963ca71860f5409a6580026de"}
Jul 17 10:02:57.425324 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:57.425305    2574 scope.go:117] "RemoveContainer" containerID="43c22adc281abd1c29cdc0edc94209b6723d53d963ca71860f5409a6580026de"
Jul 17 10:02:57.426294 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:57.426269    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mpgtb" event={"ID":"33352f89-e1a0-4192-96c7-279d9e72c734","Type":"ContainerStarted","Data":"fa3e553fe0bf4afe1c368ec84ba9f4ac48b96742efd20a716a0dd69ba0fc9583"}
Jul 17 10:02:58.433561 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:58.433528    2574 generic.go:356] "Generic (PLEG): container finished" podID="c90a0362-eb84-45f1-aa4b-ac3a37ec46f7" containerID="a86169f70b8e606b47b905900413bae9565569ddbd6462dfb162233c92c95207" exitCode=7
Jul 17 10:02:58.433989 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:58.433612    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-r7xhf" event={"ID":"c90a0362-eb84-45f1-aa4b-ac3a37ec46f7","Type":"ContainerDied","Data":"a86169f70b8e606b47b905900413bae9565569ddbd6462dfb162233c92c95207"}
Jul 17 10:02:58.433989 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:58.433666    2574 scope.go:117] "RemoveContainer" containerID="43c22adc281abd1c29cdc0edc94209b6723d53d963ca71860f5409a6580026de"
Jul 17 10:02:58.433989 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:58.433895    2574 scope.go:117] "RemoveContainer" containerID="a86169f70b8e606b47b905900413bae9565569ddbd6462dfb162233c92c95207"
Jul 17 10:02:58.434174 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:02:58.434149    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-r7xhf_odh-ai-gateway-infra(c90a0362-eb84-45f1-aa4b-ac3a37ec46f7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-r7xhf" podUID="c90a0362-eb84-45f1-aa4b-ac3a37ec46f7"
Jul 17 10:02:58.435320 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:58.435300    2574 generic.go:356] "Generic (PLEG): container finished" podID="33352f89-e1a0-4192-96c7-279d9e72c734" containerID="a6b3b757533c5792731c11c0fddf018119549aa016be021bdf16f04e400d4c9c" exitCode=6
Jul 17 10:02:58.435405 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:58.435353    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mpgtb" event={"ID":"33352f89-e1a0-4192-96c7-279d9e72c734","Type":"ContainerDied","Data":"a6b3b757533c5792731c11c0fddf018119549aa016be021bdf16f04e400d4c9c"}
Jul 17 10:02:58.435573 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:58.435558    2574 scope.go:117] "RemoveContainer" containerID="a6b3b757533c5792731c11c0fddf018119549aa016be021bdf16f04e400d4c9c"
Jul 17 10:02:59.443435 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:59.443407    2574 scope.go:117] "RemoveContainer" containerID="a86169f70b8e606b47b905900413bae9565569ddbd6462dfb162233c92c95207"
Jul 17 10:02:59.443871 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:02:59.443639    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-r7xhf_odh-ai-gateway-infra(c90a0362-eb84-45f1-aa4b-ac3a37ec46f7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-r7xhf" podUID="c90a0362-eb84-45f1-aa4b-ac3a37ec46f7"
Jul 17 10:02:59.444966 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:59.444941    2574 generic.go:356] "Generic (PLEG): container finished" podID="33352f89-e1a0-4192-96c7-279d9e72c734" containerID="76088a01ab88b3371f1548a8911d69bad8493bf20831df73f337fd039c0e5511" exitCode=6
Jul 17 10:02:59.445090 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:59.445022    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mpgtb" event={"ID":"33352f89-e1a0-4192-96c7-279d9e72c734","Type":"ContainerDied","Data":"76088a01ab88b3371f1548a8911d69bad8493bf20831df73f337fd039c0e5511"}
Jul 17 10:02:59.445090 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:59.445081    2574 scope.go:117] "RemoveContainer" containerID="a6b3b757533c5792731c11c0fddf018119549aa016be021bdf16f04e400d4c9c"
Jul 17 10:02:59.445277 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:02:59.445263    2574 scope.go:117] "RemoveContainer" containerID="76088a01ab88b3371f1548a8911d69bad8493bf20831df73f337fd039c0e5511"
Jul 17 10:02:59.445509 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:02:59.445491    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-mpgtb_odh-ai-gateway-infra(33352f89-e1a0-4192-96c7-279d9e72c734)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mpgtb" podUID="33352f89-e1a0-4192-96c7-279d9e72c734"
Jul 17 10:03:00.451985 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:00.451955    2574 scope.go:117] "RemoveContainer" containerID="76088a01ab88b3371f1548a8911d69bad8493bf20831df73f337fd039c0e5511"
Jul 17 10:03:00.452415 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:03:00.452144    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-mpgtb_odh-ai-gateway-infra(33352f89-e1a0-4192-96c7-279d9e72c734)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mpgtb" podUID="33352f89-e1a0-4192-96c7-279d9e72c734"
Jul 17 10:03:07.509365 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:07.509327    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6c5c6b98f-rlrg6_8ab844e0-7c94-4003-9e42-ada02b05d4e9/manager/0.log"
Jul 17 10:03:11.891804 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:11.891776    2574 scope.go:117] "RemoveContainer" containerID="a86169f70b8e606b47b905900413bae9565569ddbd6462dfb162233c92c95207"
Jul 17 10:03:12.526903 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:12.526874    2574 generic.go:356] "Generic (PLEG): container finished" podID="c90a0362-eb84-45f1-aa4b-ac3a37ec46f7" containerID="5dd5bdc747b550369b6e12c9a952321fd60f17367f0f907abe1991cdad5424a7" exitCode=7
Jul 17 10:03:12.527021 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:12.526955    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-r7xhf" event={"ID":"c90a0362-eb84-45f1-aa4b-ac3a37ec46f7","Type":"ContainerDied","Data":"5dd5bdc747b550369b6e12c9a952321fd60f17367f0f907abe1991cdad5424a7"}
Jul 17 10:03:12.527086 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:12.527067    2574 scope.go:117] "RemoveContainer" containerID="a86169f70b8e606b47b905900413bae9565569ddbd6462dfb162233c92c95207"
Jul 17 10:03:12.527438 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:12.527422    2574 scope.go:117] "RemoveContainer" containerID="5dd5bdc747b550369b6e12c9a952321fd60f17367f0f907abe1991cdad5424a7"
Jul 17 10:03:12.527693 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:03:12.527669    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-r7xhf_odh-ai-gateway-infra(c90a0362-eb84-45f1-aa4b-ac3a37ec46f7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-r7xhf" podUID="c90a0362-eb84-45f1-aa4b-ac3a37ec46f7"
Jul 17 10:03:13.553772 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:13.553732    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-r7xhf"]
Jul 17 10:03:13.700041 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:13.700019    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-r7xhf"
Jul 17 10:03:13.710167 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:13.710142    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c90a0362-eb84-45f1-aa4b-ac3a37ec46f7-maas-api-service-ca\") pod \"c90a0362-eb84-45f1-aa4b-ac3a37ec46f7\" (UID: \"c90a0362-eb84-45f1-aa4b-ac3a37ec46f7\") "
Jul 17 10:03:13.710536 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:13.710513    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c90a0362-eb84-45f1-aa4b-ac3a37ec46f7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c90a0362-eb84-45f1-aa4b-ac3a37ec46f7" (UID: "c90a0362-eb84-45f1-aa4b-ac3a37ec46f7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:03:13.811481 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:13.811398    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c90a0362-eb84-45f1-aa4b-ac3a37ec46f7-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:03:14.541502 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:14.541477    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-r7xhf"
Jul 17 10:03:14.541700 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:14.541519    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-r7xhf" event={"ID":"c90a0362-eb84-45f1-aa4b-ac3a37ec46f7","Type":"ContainerDied","Data":"0730d364d453d64d57259743ece54e11da6ebe7c1814c6e86f1b05b8825a3e89"}
Jul 17 10:03:14.541700 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:14.541642    2574 scope.go:117] "RemoveContainer" containerID="5dd5bdc747b550369b6e12c9a952321fd60f17367f0f907abe1991cdad5424a7"
Jul 17 10:03:14.565132 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:14.565108    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-r7xhf"]
Jul 17 10:03:14.570635 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:14.570607    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-r7xhf"]
Jul 17 10:03:14.897864 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:14.897776    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c90a0362-eb84-45f1-aa4b-ac3a37ec46f7" path="/var/lib/kubelet/pods/c90a0362-eb84-45f1-aa4b-ac3a37ec46f7/volumes"
Jul 17 10:03:15.891440 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:15.891409    2574 scope.go:117] "RemoveContainer" containerID="76088a01ab88b3371f1548a8911d69bad8493bf20831df73f337fd039c0e5511"
Jul 17 10:03:16.561444 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:16.561413    2574 generic.go:356] "Generic (PLEG): container finished" podID="33352f89-e1a0-4192-96c7-279d9e72c734" containerID="539e34de6f6756b072c542289d3f8350c770d38f8598fc166970c9bf8a0bcbce" exitCode=6
Jul 17 10:03:16.561621 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:16.561493    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mpgtb" event={"ID":"33352f89-e1a0-4192-96c7-279d9e72c734","Type":"ContainerDied","Data":"539e34de6f6756b072c542289d3f8350c770d38f8598fc166970c9bf8a0bcbce"}
Jul 17 10:03:16.561621 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:16.561586    2574 scope.go:117] "RemoveContainer" containerID="76088a01ab88b3371f1548a8911d69bad8493bf20831df73f337fd039c0e5511"
Jul 17 10:03:16.561913 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:16.561894    2574 scope.go:117] "RemoveContainer" containerID="539e34de6f6756b072c542289d3f8350c770d38f8598fc166970c9bf8a0bcbce"
Jul 17 10:03:16.562157 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:03:16.562135    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-mpgtb_odh-ai-gateway-infra(33352f89-e1a0-4192-96c7-279d9e72c734)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mpgtb" podUID="33352f89-e1a0-4192-96c7-279d9e72c734"
Jul 17 10:03:17.586746 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:17.586713    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mpgtb"]
Jul 17 10:03:17.735674 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:17.735648    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mpgtb"
Jul 17 10:03:17.747430 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:17.747405    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33352f89-e1a0-4192-96c7-279d9e72c734-maas-api-service-ca\") pod \"33352f89-e1a0-4192-96c7-279d9e72c734\" (UID: \"33352f89-e1a0-4192-96c7-279d9e72c734\") "
Jul 17 10:03:17.747722 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:17.747701    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33352f89-e1a0-4192-96c7-279d9e72c734-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "33352f89-e1a0-4192-96c7-279d9e72c734" (UID: "33352f89-e1a0-4192-96c7-279d9e72c734"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:03:17.848936 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:17.848852    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/33352f89-e1a0-4192-96c7-279d9e72c734-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:03:18.575010 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:18.574977    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mpgtb"
Jul 17 10:03:18.575218 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:18.575026    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mpgtb" event={"ID":"33352f89-e1a0-4192-96c7-279d9e72c734","Type":"ContainerDied","Data":"fa3e553fe0bf4afe1c368ec84ba9f4ac48b96742efd20a716a0dd69ba0fc9583"}
Jul 17 10:03:18.575218 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:18.575085    2574 scope.go:117] "RemoveContainer" containerID="539e34de6f6756b072c542289d3f8350c770d38f8598fc166970c9bf8a0bcbce"
Jul 17 10:03:18.600282 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:18.600231    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mpgtb"]
Jul 17 10:03:18.608358 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:18.608332    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mpgtb"]
Jul 17 10:03:18.898275 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:18.898184    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33352f89-e1a0-4192-96c7-279d9e72c734" path="/var/lib/kubelet/pods/33352f89-e1a0-4192-96c7-279d9e72c734/volumes"
Jul 17 10:03:25.213318 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:25.213275    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-kwq4b"]
Jul 17 10:03:25.213927 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:25.213905    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c90a0362-eb84-45f1-aa4b-ac3a37ec46f7" containerName="revoke-keys"
Jul 17 10:03:25.214038 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:25.213931    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c90a0362-eb84-45f1-aa4b-ac3a37ec46f7" containerName="revoke-keys"
Jul 17 10:03:25.214038 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:25.213947    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c90a0362-eb84-45f1-aa4b-ac3a37ec46f7" containerName="revoke-keys"
Jul 17 10:03:25.214038 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:25.213955    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c90a0362-eb84-45f1-aa4b-ac3a37ec46f7" containerName="revoke-keys"
Jul 17 10:03:25.214038 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:25.213968    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33352f89-e1a0-4192-96c7-279d9e72c734" containerName="revoke-keys"
Jul 17 10:03:25.214038 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:25.213976    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="33352f89-e1a0-4192-96c7-279d9e72c734" containerName="revoke-keys"
Jul 17 10:03:25.214038 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:25.213987    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c90a0362-eb84-45f1-aa4b-ac3a37ec46f7" containerName="revoke-keys"
Jul 17 10:03:25.214038 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:25.213994    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c90a0362-eb84-45f1-aa4b-ac3a37ec46f7" containerName="revoke-keys"
Jul 17 10:03:25.214038 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:25.214026    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33352f89-e1a0-4192-96c7-279d9e72c734" containerName="revoke-keys"
Jul 17 10:03:25.214038 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:25.214034    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="33352f89-e1a0-4192-96c7-279d9e72c734" containerName="revoke-keys"
Jul 17 10:03:25.214490 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:25.214149    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c90a0362-eb84-45f1-aa4b-ac3a37ec46f7" containerName="revoke-keys"
Jul 17 10:03:25.214490 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:25.214165    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c90a0362-eb84-45f1-aa4b-ac3a37ec46f7" containerName="revoke-keys"
Jul 17 10:03:25.214490 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:25.214175    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c90a0362-eb84-45f1-aa4b-ac3a37ec46f7" containerName="revoke-keys"
Jul 17 10:03:25.214490 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:25.214186    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="33352f89-e1a0-4192-96c7-279d9e72c734" containerName="revoke-keys"
Jul 17 10:03:25.214490 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:25.214196    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="33352f89-e1a0-4192-96c7-279d9e72c734" containerName="revoke-keys"
Jul 17 10:03:25.214490 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:25.214206    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="33352f89-e1a0-4192-96c7-279d9e72c734" containerName="revoke-keys"
Jul 17 10:03:25.219200 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:25.219175    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-kwq4b"
Jul 17 10:03:25.222315 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:25.222291    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:03:25.225268 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:25.225224    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-kwq4b"]
Jul 17 10:03:25.320608 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:25.320575    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-kwq4b\" (UID: \"c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-kwq4b"
Jul 17 10:03:25.421614 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:25.421580    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-kwq4b\" (UID: \"c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-kwq4b"
Jul 17 10:03:25.422278 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:25.422225    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-kwq4b\" (UID: \"c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-kwq4b"
Jul 17 10:03:25.534088 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:25.534001    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-kwq4b"
Jul 17 10:03:25.674957 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:25.674920    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-kwq4b"]
Jul 17 10:03:25.676497 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:03:25.676465    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1f4ddc5_6f94_496b_970b_46a5ee7b0b5b.slice/crio-089f7e1a62d4f31344cd5cc21ebb8701500a2c9d7c20b1ed46cd51555be674f4 WatchSource:0}: Error finding container 089f7e1a62d4f31344cd5cc21ebb8701500a2c9d7c20b1ed46cd51555be674f4: Status 404 returned error can't find the container with id 089f7e1a62d4f31344cd5cc21ebb8701500a2c9d7c20b1ed46cd51555be674f4
Jul 17 10:03:26.623835 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:26.623748    2574 generic.go:356] "Generic (PLEG): container finished" podID="c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b" containerID="e54d2254a1a3ed1cae527b8030afd720e64bf559b9d987838cf599575c2c02ba" exitCode=7
Jul 17 10:03:26.624200 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:26.623838    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-kwq4b" event={"ID":"c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b","Type":"ContainerDied","Data":"e54d2254a1a3ed1cae527b8030afd720e64bf559b9d987838cf599575c2c02ba"}
Jul 17 10:03:26.624200 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:26.623898    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-kwq4b" event={"ID":"c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b","Type":"ContainerStarted","Data":"089f7e1a62d4f31344cd5cc21ebb8701500a2c9d7c20b1ed46cd51555be674f4"}
Jul 17 10:03:26.624200 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:26.624023    2574 scope.go:117] "RemoveContainer" containerID="e54d2254a1a3ed1cae527b8030afd720e64bf559b9d987838cf599575c2c02ba"
Jul 17 10:03:27.630989 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:27.630901    2574 generic.go:356] "Generic (PLEG): container finished" podID="c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b" containerID="27465d51a452e4492b6c3bfd02966066b556cc5c064a1781d3ace07bdcb3dce1" exitCode=7
Jul 17 10:03:27.630989 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:27.630976    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-kwq4b" event={"ID":"c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b","Type":"ContainerDied","Data":"27465d51a452e4492b6c3bfd02966066b556cc5c064a1781d3ace07bdcb3dce1"}
Jul 17 10:03:27.632002 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:27.631052    2574 scope.go:117] "RemoveContainer" containerID="e54d2254a1a3ed1cae527b8030afd720e64bf559b9d987838cf599575c2c02ba"
Jul 17 10:03:27.632073 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:27.632063    2574 scope.go:117] "RemoveContainer" containerID="27465d51a452e4492b6c3bfd02966066b556cc5c064a1781d3ace07bdcb3dce1"
Jul 17 10:03:27.632392 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:03:27.632370    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-kwq4b_odh-ai-gateway-infra(c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-kwq4b" podUID="c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b"
Jul 17 10:03:27.946760 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:27.946718    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-tf98q"]
Jul 17 10:03:27.947232 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:27.947214    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33352f89-e1a0-4192-96c7-279d9e72c734" containerName="revoke-keys"
Jul 17 10:03:27.947365 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:27.947322    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="33352f89-e1a0-4192-96c7-279d9e72c734" containerName="revoke-keys"
Jul 17 10:03:27.951495 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:27.951472    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-tf98q"
Jul 17 10:03:27.957862 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:27.957834    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-tf98q"]
Jul 17 10:03:28.050460 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:28.050429    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/578fd716-b523-4ac5-911a-36fa8997d6bf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-tf98q\" (UID: \"578fd716-b523-4ac5-911a-36fa8997d6bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-tf98q"
Jul 17 10:03:28.151750 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:28.151713    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/578fd716-b523-4ac5-911a-36fa8997d6bf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-tf98q\" (UID: \"578fd716-b523-4ac5-911a-36fa8997d6bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-tf98q"
Jul 17 10:03:28.152341 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:28.152324    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/578fd716-b523-4ac5-911a-36fa8997d6bf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-tf98q\" (UID: \"578fd716-b523-4ac5-911a-36fa8997d6bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-tf98q"
Jul 17 10:03:28.263581 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:28.263506    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-tf98q"
Jul 17 10:03:28.400840 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:28.400804    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-tf98q"]
Jul 17 10:03:28.402754 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:03:28.402723    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod578fd716_b523_4ac5_911a_36fa8997d6bf.slice/crio-1f1d2c0eadf7f3e63373a54ceb2bdc05403c08178b3e089a201b15692770283d WatchSource:0}: Error finding container 1f1d2c0eadf7f3e63373a54ceb2bdc05403c08178b3e089a201b15692770283d: Status 404 returned error can't find the container with id 1f1d2c0eadf7f3e63373a54ceb2bdc05403c08178b3e089a201b15692770283d
Jul 17 10:03:28.638422 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:28.638400    2574 scope.go:117] "RemoveContainer" containerID="27465d51a452e4492b6c3bfd02966066b556cc5c064a1781d3ace07bdcb3dce1"
Jul 17 10:03:28.638805 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:03:28.638661    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-kwq4b_odh-ai-gateway-infra(c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-kwq4b" podUID="c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b"
Jul 17 10:03:28.639776 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:28.639747    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-tf98q" event={"ID":"578fd716-b523-4ac5-911a-36fa8997d6bf","Type":"ContainerStarted","Data":"80bb0f12ab52ed8bedd616f3ca701fbf251d378918cd8d625eccc5264a99e4c5"}
Jul 17 10:03:28.639895 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:28.639797    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-tf98q" event={"ID":"578fd716-b523-4ac5-911a-36fa8997d6bf","Type":"ContainerStarted","Data":"1f1d2c0eadf7f3e63373a54ceb2bdc05403c08178b3e089a201b15692770283d"}
Jul 17 10:03:28.670729 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:28.670682    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-tf98q" podStartSLOduration=1.6706708369999999 podStartE2EDuration="1.670670837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:03:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:03:28.669435158 +0000 UTC m=+2206.252073534" watchObservedRunningTime="2026-07-17 10:03:28.670670837 +0000 UTC m=+2206.253309252"
Jul 17 10:03:29.647186 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:29.647150    2574 generic.go:356] "Generic (PLEG): container finished" podID="578fd716-b523-4ac5-911a-36fa8997d6bf" containerID="80bb0f12ab52ed8bedd616f3ca701fbf251d378918cd8d625eccc5264a99e4c5" exitCode=6
Jul 17 10:03:29.647616 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:29.647262    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-tf98q" event={"ID":"578fd716-b523-4ac5-911a-36fa8997d6bf","Type":"ContainerDied","Data":"80bb0f12ab52ed8bedd616f3ca701fbf251d378918cd8d625eccc5264a99e4c5"}
Jul 17 10:03:29.647616 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:29.647451    2574 scope.go:117] "RemoveContainer" containerID="80bb0f12ab52ed8bedd616f3ca701fbf251d378918cd8d625eccc5264a99e4c5"
Jul 17 10:03:30.654278 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:30.654166    2574 generic.go:356] "Generic (PLEG): container finished" podID="578fd716-b523-4ac5-911a-36fa8997d6bf" containerID="7b829d5119331f2fd2cb6aa45f0005129af2416ef0aa355e3e1c6a1f193a1b9d" exitCode=6
Jul 17 10:03:30.654738 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:30.654271    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-tf98q" event={"ID":"578fd716-b523-4ac5-911a-36fa8997d6bf","Type":"ContainerDied","Data":"7b829d5119331f2fd2cb6aa45f0005129af2416ef0aa355e3e1c6a1f193a1b9d"}
Jul 17 10:03:30.654738 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:30.654340    2574 scope.go:117] "RemoveContainer" containerID="80bb0f12ab52ed8bedd616f3ca701fbf251d378918cd8d625eccc5264a99e4c5"
Jul 17 10:03:30.654738 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:30.654560    2574 scope.go:117] "RemoveContainer" containerID="7b829d5119331f2fd2cb6aa45f0005129af2416ef0aa355e3e1c6a1f193a1b9d"
Jul 17 10:03:30.654875 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:03:30.654770    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-tf98q_odh-ai-gateway-infra(578fd716-b523-4ac5-911a-36fa8997d6bf)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-tf98q" podUID="578fd716-b523-4ac5-911a-36fa8997d6bf"
Jul 17 10:03:31.661232 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:31.661203    2574 scope.go:117] "RemoveContainer" containerID="7b829d5119331f2fd2cb6aa45f0005129af2416ef0aa355e3e1c6a1f193a1b9d"
Jul 17 10:03:31.661621 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:03:31.661435    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-tf98q_odh-ai-gateway-infra(578fd716-b523-4ac5-911a-36fa8997d6bf)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-tf98q" podUID="578fd716-b523-4ac5-911a-36fa8997d6bf"
Jul 17 10:03:39.891858 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:39.891825    2574 scope.go:117] "RemoveContainer" containerID="27465d51a452e4492b6c3bfd02966066b556cc5c064a1781d3ace07bdcb3dce1"
Jul 17 10:03:40.714194 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:40.714159    2574 generic.go:356] "Generic (PLEG): container finished" podID="c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b" containerID="d1caca7ebdd6a2e9e45e4811475d528916b4df85acbaf33ce7b75f9bd243418d" exitCode=7
Jul 17 10:03:40.714407 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:40.714233    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-kwq4b" event={"ID":"c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b","Type":"ContainerDied","Data":"d1caca7ebdd6a2e9e45e4811475d528916b4df85acbaf33ce7b75f9bd243418d"}
Jul 17 10:03:40.714407 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:40.714349    2574 scope.go:117] "RemoveContainer" containerID="27465d51a452e4492b6c3bfd02966066b556cc5c064a1781d3ace07bdcb3dce1"
Jul 17 10:03:40.714651 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:40.714633    2574 scope.go:117] "RemoveContainer" containerID="d1caca7ebdd6a2e9e45e4811475d528916b4df85acbaf33ce7b75f9bd243418d"
Jul 17 10:03:40.714924 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:03:40.714892    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-kwq4b_odh-ai-gateway-infra(c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-kwq4b" podUID="c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b"
Jul 17 10:03:41.740706 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:41.740672    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-kwq4b"]
Jul 17 10:03:41.872821 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:41.872792    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-kwq4b"
Jul 17 10:03:41.980655 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:41.980617    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b-maas-api-service-ca\") pod \"c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b\" (UID: \"c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b\") "
Jul 17 10:03:41.980998 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:41.980972    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b" (UID: "c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:03:42.081849 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:42.081752    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:03:42.727567 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:42.727541    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-kwq4b"
Jul 17 10:03:42.727764 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:42.727577    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-kwq4b" event={"ID":"c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b","Type":"ContainerDied","Data":"089f7e1a62d4f31344cd5cc21ebb8701500a2c9d7c20b1ed46cd51555be674f4"}
Jul 17 10:03:42.727764 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:42.727658    2574 scope.go:117] "RemoveContainer" containerID="d1caca7ebdd6a2e9e45e4811475d528916b4df85acbaf33ce7b75f9bd243418d"
Jul 17 10:03:42.752859 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:42.752833    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-kwq4b"]
Jul 17 10:03:42.760457 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:42.760435    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-kwq4b"]
Jul 17 10:03:42.898101 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:42.898067    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b" path="/var/lib/kubelet/pods/c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b/volumes"
Jul 17 10:03:45.848653 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:45.848619    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh"]
Jul 17 10:03:45.849080 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:45.849041    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b" containerName="revoke-keys"
Jul 17 10:03:45.849080 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:45.849053    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b" containerName="revoke-keys"
Jul 17 10:03:45.849167 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:45.849083    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b" containerName="revoke-keys"
Jul 17 10:03:45.849167 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:45.849090    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b" containerName="revoke-keys"
Jul 17 10:03:45.849167 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:45.849161    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b" containerName="revoke-keys"
Jul 17 10:03:45.849313 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:45.849173    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b" containerName="revoke-keys"
Jul 17 10:03:45.853817 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:45.853799    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh"
Jul 17 10:03:45.861055 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:45.861031    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh"]
Jul 17 10:03:46.021654 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:46.021620    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16442b8f-3ab3-4ca5-972b-16668d8640c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh\" (UID: \"16442b8f-3ab3-4ca5-972b-16668d8640c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh"
Jul 17 10:03:46.122566 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:46.122467    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16442b8f-3ab3-4ca5-972b-16668d8640c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh\" (UID: \"16442b8f-3ab3-4ca5-972b-16668d8640c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh"
Jul 17 10:03:46.123126 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:46.123100    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16442b8f-3ab3-4ca5-972b-16668d8640c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh\" (UID: \"16442b8f-3ab3-4ca5-972b-16668d8640c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh"
Jul 17 10:03:46.166481 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:46.166451    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh"
Jul 17 10:03:46.300330 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:46.300300    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh"]
Jul 17 10:03:46.302226 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:03:46.302185    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16442b8f_3ab3_4ca5_972b_16668d8640c1.slice/crio-3692356fcbc385b4acf6605154b709881342e06430272e18a72656dbf720506b WatchSource:0}: Error finding container 3692356fcbc385b4acf6605154b709881342e06430272e18a72656dbf720506b: Status 404 returned error can't find the container with id 3692356fcbc385b4acf6605154b709881342e06430272e18a72656dbf720506b
Jul 17 10:03:46.749737 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:46.749702    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh" event={"ID":"16442b8f-3ab3-4ca5-972b-16668d8640c1","Type":"ContainerStarted","Data":"59fc5544764660c04c3c0bd2a257e8e2d9f1b2676b4fc4a5304111020a681d51"}
Jul 17 10:03:46.749905 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:46.749752    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh" event={"ID":"16442b8f-3ab3-4ca5-972b-16668d8640c1","Type":"ContainerStarted","Data":"3692356fcbc385b4acf6605154b709881342e06430272e18a72656dbf720506b"}
Jul 17 10:03:46.766686 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:46.766637    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh" podStartSLOduration=1.7666232910000002 podStartE2EDuration="1.766623291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:03:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:03:46.765996575 +0000 UTC m=+2224.348634955" watchObservedRunningTime="2026-07-17 10:03:46.766623291 +0000 UTC m=+2224.349261666"
Jul 17 10:03:46.892091 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:46.892006    2574 scope.go:117] "RemoveContainer" containerID="7b829d5119331f2fd2cb6aa45f0005129af2416ef0aa355e3e1c6a1f193a1b9d"
Jul 17 10:03:47.757403 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:47.757366    2574 generic.go:356] "Generic (PLEG): container finished" podID="578fd716-b523-4ac5-911a-36fa8997d6bf" containerID="b224391f7355dbf19e111e675bd75dac0c4fd58cb4617576842f4c2e9ca5b8f1" exitCode=6
Jul 17 10:03:47.757576 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:47.757437    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-tf98q" event={"ID":"578fd716-b523-4ac5-911a-36fa8997d6bf","Type":"ContainerDied","Data":"b224391f7355dbf19e111e675bd75dac0c4fd58cb4617576842f4c2e9ca5b8f1"}
Jul 17 10:03:47.757576 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:47.757494    2574 scope.go:117] "RemoveContainer" containerID="7b829d5119331f2fd2cb6aa45f0005129af2416ef0aa355e3e1c6a1f193a1b9d"
Jul 17 10:03:47.757908 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:47.757886    2574 scope.go:117] "RemoveContainer" containerID="b224391f7355dbf19e111e675bd75dac0c4fd58cb4617576842f4c2e9ca5b8f1"
Jul 17 10:03:47.758185 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:03:47.758161    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-tf98q_odh-ai-gateway-infra(578fd716-b523-4ac5-911a-36fa8997d6bf)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-tf98q" podUID="578fd716-b523-4ac5-911a-36fa8997d6bf"
Jul 17 10:03:47.759126 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:47.759103    2574 generic.go:356] "Generic (PLEG): container finished" podID="16442b8f-3ab3-4ca5-972b-16668d8640c1" containerID="59fc5544764660c04c3c0bd2a257e8e2d9f1b2676b4fc4a5304111020a681d51" exitCode=6
Jul 17 10:03:47.759229 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:47.759181    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh" event={"ID":"16442b8f-3ab3-4ca5-972b-16668d8640c1","Type":"ContainerDied","Data":"59fc5544764660c04c3c0bd2a257e8e2d9f1b2676b4fc4a5304111020a681d51"}
Jul 17 10:03:47.759390 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:47.759375    2574 scope.go:117] "RemoveContainer" containerID="59fc5544764660c04c3c0bd2a257e8e2d9f1b2676b4fc4a5304111020a681d51"
Jul 17 10:03:48.768194 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:48.768161    2574 generic.go:356] "Generic (PLEG): container finished" podID="16442b8f-3ab3-4ca5-972b-16668d8640c1" containerID="259e1e3168034ee8f625a0b48a0937bd1dcbd95f87bea62a9aa8bf444a3701e8" exitCode=6
Jul 17 10:03:48.768715 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:48.768193    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh" event={"ID":"16442b8f-3ab3-4ca5-972b-16668d8640c1","Type":"ContainerDied","Data":"259e1e3168034ee8f625a0b48a0937bd1dcbd95f87bea62a9aa8bf444a3701e8"}
Jul 17 10:03:48.768715 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:48.768286    2574 scope.go:117] "RemoveContainer" containerID="59fc5544764660c04c3c0bd2a257e8e2d9f1b2676b4fc4a5304111020a681d51"
Jul 17 10:03:48.768715 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:48.768553    2574 scope.go:117] "RemoveContainer" containerID="259e1e3168034ee8f625a0b48a0937bd1dcbd95f87bea62a9aa8bf444a3701e8"
Jul 17 10:03:48.768874 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:03:48.768824    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh_odh-ai-gateway-infra(16442b8f-3ab3-4ca5-972b-16668d8640c1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh" podUID="16442b8f-3ab3-4ca5-972b-16668d8640c1"
Jul 17 10:03:48.783714 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:48.783691    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-tf98q"]
Jul 17 10:03:48.931601 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:48.931579    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-tf98q"
Jul 17 10:03:49.052992 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:49.052907    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/578fd716-b523-4ac5-911a-36fa8997d6bf-maas-api-service-ca\") pod \"578fd716-b523-4ac5-911a-36fa8997d6bf\" (UID: \"578fd716-b523-4ac5-911a-36fa8997d6bf\") "
Jul 17 10:03:49.053329 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:49.053301    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/578fd716-b523-4ac5-911a-36fa8997d6bf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "578fd716-b523-4ac5-911a-36fa8997d6bf" (UID: "578fd716-b523-4ac5-911a-36fa8997d6bf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:03:49.153779 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:49.153739    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/578fd716-b523-4ac5-911a-36fa8997d6bf-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:03:49.775143 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:49.775111    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-tf98q"
Jul 17 10:03:49.775595 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:49.775110    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-tf98q" event={"ID":"578fd716-b523-4ac5-911a-36fa8997d6bf","Type":"ContainerDied","Data":"1f1d2c0eadf7f3e63373a54ceb2bdc05403c08178b3e089a201b15692770283d"}
Jul 17 10:03:49.775595 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:49.775280    2574 scope.go:117] "RemoveContainer" containerID="b224391f7355dbf19e111e675bd75dac0c4fd58cb4617576842f4c2e9ca5b8f1"
Jul 17 10:03:49.777510 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:49.777492    2574 scope.go:117] "RemoveContainer" containerID="259e1e3168034ee8f625a0b48a0937bd1dcbd95f87bea62a9aa8bf444a3701e8"
Jul 17 10:03:49.777683 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:03:49.777664    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh_odh-ai-gateway-infra(16442b8f-3ab3-4ca5-972b-16668d8640c1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh" podUID="16442b8f-3ab3-4ca5-972b-16668d8640c1"
Jul 17 10:03:49.806101 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:49.806073    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-tf98q"]
Jul 17 10:03:49.809800 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:49.809778    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-tf98q"]
Jul 17 10:03:50.898201 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:50.898168    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="578fd716-b523-4ac5-911a-36fa8997d6bf" path="/var/lib/kubelet/pods/578fd716-b523-4ac5-911a-36fa8997d6bf/volumes"
Jul 17 10:03:56.806261 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:56.806185    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x"]
Jul 17 10:03:56.806836 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:56.806675    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="578fd716-b523-4ac5-911a-36fa8997d6bf" containerName="revoke-keys"
Jul 17 10:03:56.806836 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:56.806687    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="578fd716-b523-4ac5-911a-36fa8997d6bf" containerName="revoke-keys"
Jul 17 10:03:56.806836 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:56.806701    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b" containerName="revoke-keys"
Jul 17 10:03:56.806836 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:56.806706    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b" containerName="revoke-keys"
Jul 17 10:03:56.806836 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:56.806713    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="578fd716-b523-4ac5-911a-36fa8997d6bf" containerName="revoke-keys"
Jul 17 10:03:56.806836 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:56.806718    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="578fd716-b523-4ac5-911a-36fa8997d6bf" containerName="revoke-keys"
Jul 17 10:03:56.806836 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:56.806726    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="578fd716-b523-4ac5-911a-36fa8997d6bf" containerName="revoke-keys"
Jul 17 10:03:56.806836 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:56.806731    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="578fd716-b523-4ac5-911a-36fa8997d6bf" containerName="revoke-keys"
Jul 17 10:03:56.806836 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:56.806789    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="578fd716-b523-4ac5-911a-36fa8997d6bf" containerName="revoke-keys"
Jul 17 10:03:56.806836 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:56.806796    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="578fd716-b523-4ac5-911a-36fa8997d6bf" containerName="revoke-keys"
Jul 17 10:03:56.806836 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:56.806805    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1f4ddc5-6f94-496b-970b-46a5ee7b0b5b" containerName="revoke-keys"
Jul 17 10:03:56.810831 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:56.810810    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x"
Jul 17 10:03:56.818684 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:56.818648    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x"]
Jul 17 10:03:56.827285 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:56.827200    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5e8923d-9adf-4a7e-b1ce-bded023d6b3c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x\" (UID: \"c5e8923d-9adf-4a7e-b1ce-bded023d6b3c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x"
Jul 17 10:03:56.928755 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:56.928721    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5e8923d-9adf-4a7e-b1ce-bded023d6b3c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x\" (UID: \"c5e8923d-9adf-4a7e-b1ce-bded023d6b3c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x"
Jul 17 10:03:56.929375 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:56.929351    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5e8923d-9adf-4a7e-b1ce-bded023d6b3c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x\" (UID: \"c5e8923d-9adf-4a7e-b1ce-bded023d6b3c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x"
Jul 17 10:03:57.125221 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:57.125134    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x"
Jul 17 10:03:57.264215 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:57.264179    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x"]
Jul 17 10:03:57.267431 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:03:57.267386    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5e8923d_9adf_4a7e_b1ce_bded023d6b3c.slice/crio-cb6bed70e9ed8bbfdeca57a086dd3c178ae01180ade5b861e602aaaa3599a9a0 WatchSource:0}: Error finding container cb6bed70e9ed8bbfdeca57a086dd3c178ae01180ade5b861e602aaaa3599a9a0: Status 404 returned error can't find the container with id cb6bed70e9ed8bbfdeca57a086dd3c178ae01180ade5b861e602aaaa3599a9a0
Jul 17 10:03:57.829427 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:57.829338    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x" event={"ID":"c5e8923d-9adf-4a7e-b1ce-bded023d6b3c","Type":"ContainerStarted","Data":"3764522a9eb8e13d404b9f604c5252bb4f7f2ec2c75a8088e460b42c7dc85d61"}
Jul 17 10:03:57.829427 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:57.829393    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x" event={"ID":"c5e8923d-9adf-4a7e-b1ce-bded023d6b3c","Type":"ContainerStarted","Data":"cb6bed70e9ed8bbfdeca57a086dd3c178ae01180ade5b861e602aaaa3599a9a0"}
Jul 17 10:03:57.851668 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:57.851620    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x" podStartSLOduration=1.8516041429999999 podStartE2EDuration="1.851604143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:03:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:03:57.84876219 +0000 UTC m=+2235.431400564" watchObservedRunningTime="2026-07-17 10:03:57.851604143 +0000 UTC m=+2235.434242518"
Jul 17 10:03:58.836568 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:58.836535    2574 generic.go:356] "Generic (PLEG): container finished" podID="c5e8923d-9adf-4a7e-b1ce-bded023d6b3c" containerID="3764522a9eb8e13d404b9f604c5252bb4f7f2ec2c75a8088e460b42c7dc85d61" exitCode=6
Jul 17 10:03:58.837063 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:58.836620    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x" event={"ID":"c5e8923d-9adf-4a7e-b1ce-bded023d6b3c","Type":"ContainerDied","Data":"3764522a9eb8e13d404b9f604c5252bb4f7f2ec2c75a8088e460b42c7dc85d61"}
Jul 17 10:03:58.837063 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:58.836850    2574 scope.go:117] "RemoveContainer" containerID="3764522a9eb8e13d404b9f604c5252bb4f7f2ec2c75a8088e460b42c7dc85d61"
Jul 17 10:03:59.843612 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:59.843576    2574 generic.go:356] "Generic (PLEG): container finished" podID="c5e8923d-9adf-4a7e-b1ce-bded023d6b3c" containerID="d251a5e16b921ff5debfcb145e5a4e7f8d493dc2defd75b9c48d8ac28b43f39a" exitCode=6
Jul 17 10:03:59.844103 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:59.843661    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x" event={"ID":"c5e8923d-9adf-4a7e-b1ce-bded023d6b3c","Type":"ContainerDied","Data":"d251a5e16b921ff5debfcb145e5a4e7f8d493dc2defd75b9c48d8ac28b43f39a"}
Jul 17 10:03:59.844103 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:59.843738    2574 scope.go:117] "RemoveContainer" containerID="3764522a9eb8e13d404b9f604c5252bb4f7f2ec2c75a8088e460b42c7dc85d61"
Jul 17 10:03:59.844103 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:03:59.843912    2574 scope.go:117] "RemoveContainer" containerID="d251a5e16b921ff5debfcb145e5a4e7f8d493dc2defd75b9c48d8ac28b43f39a"
Jul 17 10:03:59.844293 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:03:59.844178    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x_odh-ai-gateway-infra(c5e8923d-9adf-4a7e-b1ce-bded023d6b3c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x" podUID="c5e8923d-9adf-4a7e-b1ce-bded023d6b3c"
Jul 17 10:04:00.850856 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:00.850827    2574 scope.go:117] "RemoveContainer" containerID="d251a5e16b921ff5debfcb145e5a4e7f8d493dc2defd75b9c48d8ac28b43f39a"
Jul 17 10:04:00.851270 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:04:00.851062    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x_odh-ai-gateway-infra(c5e8923d-9adf-4a7e-b1ce-bded023d6b3c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x" podUID="c5e8923d-9adf-4a7e-b1ce-bded023d6b3c"
Jul 17 10:04:00.891875 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:00.891850    2574 scope.go:117] "RemoveContainer" containerID="259e1e3168034ee8f625a0b48a0937bd1dcbd95f87bea62a9aa8bf444a3701e8"
Jul 17 10:04:01.857931 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:01.857842    2574 generic.go:356] "Generic (PLEG): container finished" podID="16442b8f-3ab3-4ca5-972b-16668d8640c1" containerID="2b99100d981d34fd3a99635742b41d8e459c93aed8e4b1fb3ffd79f3ab27ded8" exitCode=6
Jul 17 10:04:01.857931 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:01.857917    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh" event={"ID":"16442b8f-3ab3-4ca5-972b-16668d8640c1","Type":"ContainerDied","Data":"2b99100d981d34fd3a99635742b41d8e459c93aed8e4b1fb3ffd79f3ab27ded8"}
Jul 17 10:04:01.858407 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:01.857980    2574 scope.go:117] "RemoveContainer" containerID="259e1e3168034ee8f625a0b48a0937bd1dcbd95f87bea62a9aa8bf444a3701e8"
Jul 17 10:04:01.858407 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:01.858329    2574 scope.go:117] "RemoveContainer" containerID="2b99100d981d34fd3a99635742b41d8e459c93aed8e4b1fb3ffd79f3ab27ded8"
Jul 17 10:04:01.858577 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:04:01.858556    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh_odh-ai-gateway-infra(16442b8f-3ab3-4ca5-972b-16668d8640c1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh" podUID="16442b8f-3ab3-4ca5-972b-16668d8640c1"
Jul 17 10:04:02.883657 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:02.883618    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh"]
Jul 17 10:04:03.034604 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:03.034580    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh"
Jul 17 10:04:03.086342 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:03.086312    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16442b8f-3ab3-4ca5-972b-16668d8640c1-maas-api-service-ca\") pod \"16442b8f-3ab3-4ca5-972b-16668d8640c1\" (UID: \"16442b8f-3ab3-4ca5-972b-16668d8640c1\") "
Jul 17 10:04:03.086647 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:03.086625    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16442b8f-3ab3-4ca5-972b-16668d8640c1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "16442b8f-3ab3-4ca5-972b-16668d8640c1" (UID: "16442b8f-3ab3-4ca5-972b-16668d8640c1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:04:03.187694 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:03.187661    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16442b8f-3ab3-4ca5-972b-16668d8640c1-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:04:03.872598 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:03.872567    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh"
Jul 17 10:04:03.872767 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:03.872611    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh" event={"ID":"16442b8f-3ab3-4ca5-972b-16668d8640c1","Type":"ContainerDied","Data":"3692356fcbc385b4acf6605154b709881342e06430272e18a72656dbf720506b"}
Jul 17 10:04:03.872767 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:03.872681    2574 scope.go:117] "RemoveContainer" containerID="2b99100d981d34fd3a99635742b41d8e459c93aed8e4b1fb3ffd79f3ab27ded8"
Jul 17 10:04:03.896562 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:03.896529    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh"]
Jul 17 10:04:03.901892 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:03.901869    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bf6nh"]
Jul 17 10:04:04.897393 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:04.897359    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16442b8f-3ab3-4ca5-972b-16668d8640c1" path="/var/lib/kubelet/pods/16442b8f-3ab3-4ca5-972b-16668d8640c1/volumes"
Jul 17 10:04:14.892295 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:14.892261    2574 scope.go:117] "RemoveContainer" containerID="d251a5e16b921ff5debfcb145e5a4e7f8d493dc2defd75b9c48d8ac28b43f39a"
Jul 17 10:04:15.278810 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:15.278778    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-89knf"]
Jul 17 10:04:15.279350 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:15.279326    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16442b8f-3ab3-4ca5-972b-16668d8640c1" containerName="revoke-keys"
Jul 17 10:04:15.279350 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:15.279351    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="16442b8f-3ab3-4ca5-972b-16668d8640c1" containerName="revoke-keys"
Jul 17 10:04:15.279476 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:15.279374    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16442b8f-3ab3-4ca5-972b-16668d8640c1" containerName="revoke-keys"
Jul 17 10:04:15.279476 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:15.279379    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="16442b8f-3ab3-4ca5-972b-16668d8640c1" containerName="revoke-keys"
Jul 17 10:04:15.279476 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:15.279393    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16442b8f-3ab3-4ca5-972b-16668d8640c1" containerName="revoke-keys"
Jul 17 10:04:15.279476 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:15.279398    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="16442b8f-3ab3-4ca5-972b-16668d8640c1" containerName="revoke-keys"
Jul 17 10:04:15.279616 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:15.279491    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="16442b8f-3ab3-4ca5-972b-16668d8640c1" containerName="revoke-keys"
Jul 17 10:04:15.279616 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:15.279500    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="16442b8f-3ab3-4ca5-972b-16668d8640c1" containerName="revoke-keys"
Jul 17 10:04:15.279616 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:15.279507    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="578fd716-b523-4ac5-911a-36fa8997d6bf" containerName="revoke-keys"
Jul 17 10:04:15.282745 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:15.282729    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-89knf"
Jul 17 10:04:15.296561 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:15.296536    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-89knf"]
Jul 17 10:04:15.405459 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:15.405423    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d37bb2e2-ffd7-4004-876d-f209975bae87-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-89knf\" (UID: \"d37bb2e2-ffd7-4004-876d-f209975bae87\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-89knf"
Jul 17 10:04:15.506556 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:15.506460    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d37bb2e2-ffd7-4004-876d-f209975bae87-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-89knf\" (UID: \"d37bb2e2-ffd7-4004-876d-f209975bae87\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-89knf"
Jul 17 10:04:15.507080 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:15.507058    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d37bb2e2-ffd7-4004-876d-f209975bae87-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-89knf\" (UID: \"d37bb2e2-ffd7-4004-876d-f209975bae87\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-89knf"
Jul 17 10:04:15.593077 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:15.593043    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-89knf"
Jul 17 10:04:15.940489 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:15.940456    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-89knf"]
Jul 17 10:04:15.941294 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:15.941263    2574 generic.go:356] "Generic (PLEG): container finished" podID="c5e8923d-9adf-4a7e-b1ce-bded023d6b3c" containerID="a6a8eee99237b8d1ec15df3bfd834af32a7e7c8ecfdff144ef17b99d69dc6e7f" exitCode=6
Jul 17 10:04:15.941294 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:15.941276    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x" event={"ID":"c5e8923d-9adf-4a7e-b1ce-bded023d6b3c","Type":"ContainerDied","Data":"a6a8eee99237b8d1ec15df3bfd834af32a7e7c8ecfdff144ef17b99d69dc6e7f"}
Jul 17 10:04:15.941475 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:15.941345    2574 scope.go:117] "RemoveContainer" containerID="d251a5e16b921ff5debfcb145e5a4e7f8d493dc2defd75b9c48d8ac28b43f39a"
Jul 17 10:04:15.941664 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:15.941644    2574 scope.go:117] "RemoveContainer" containerID="a6a8eee99237b8d1ec15df3bfd834af32a7e7c8ecfdff144ef17b99d69dc6e7f"
Jul 17 10:04:15.941806 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:04:15.941735    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd37bb2e2_ffd7_4004_876d_f209975bae87.slice/crio-0bf5e7ec81f50056b6833ccf7056fe62d8299f3abe64a67f7f1a23651e1cac80 WatchSource:0}: Error finding container 0bf5e7ec81f50056b6833ccf7056fe62d8299f3abe64a67f7f1a23651e1cac80: Status 404 returned error can't find the container with id 0bf5e7ec81f50056b6833ccf7056fe62d8299f3abe64a67f7f1a23651e1cac80
Jul 17 10:04:15.941942 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:04:15.941917    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x_odh-ai-gateway-infra(c5e8923d-9adf-4a7e-b1ce-bded023d6b3c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x" podUID="c5e8923d-9adf-4a7e-b1ce-bded023d6b3c"
Jul 17 10:04:16.948202 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:16.948155    2574 generic.go:356] "Generic (PLEG): container finished" podID="d37bb2e2-ffd7-4004-876d-f209975bae87" containerID="77fb99600eb96f6c71aac109d30f0f9cdeb64c4635e88c4e116ce367c8ad9611" exitCode=7
Jul 17 10:04:16.948667 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:16.948215    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-89knf" event={"ID":"d37bb2e2-ffd7-4004-876d-f209975bae87","Type":"ContainerDied","Data":"77fb99600eb96f6c71aac109d30f0f9cdeb64c4635e88c4e116ce367c8ad9611"}
Jul 17 10:04:16.948667 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:16.948278    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-89knf" event={"ID":"d37bb2e2-ffd7-4004-876d-f209975bae87","Type":"ContainerStarted","Data":"0bf5e7ec81f50056b6833ccf7056fe62d8299f3abe64a67f7f1a23651e1cac80"}
Jul 17 10:04:16.948667 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:16.948474    2574 scope.go:117] "RemoveContainer" containerID="77fb99600eb96f6c71aac109d30f0f9cdeb64c4635e88c4e116ce367c8ad9611"
Jul 17 10:04:16.969703 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:16.969674    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x"]
Jul 17 10:04:17.123370 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:17.123342    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x"
Jul 17 10:04:17.221731 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:17.221703    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5e8923d-9adf-4a7e-b1ce-bded023d6b3c-maas-api-service-ca\") pod \"c5e8923d-9adf-4a7e-b1ce-bded023d6b3c\" (UID: \"c5e8923d-9adf-4a7e-b1ce-bded023d6b3c\") "
Jul 17 10:04:17.222019 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:17.221999    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5e8923d-9adf-4a7e-b1ce-bded023d6b3c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c5e8923d-9adf-4a7e-b1ce-bded023d6b3c" (UID: "c5e8923d-9adf-4a7e-b1ce-bded023d6b3c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:04:17.322966 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:17.322932    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5e8923d-9adf-4a7e-b1ce-bded023d6b3c-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:04:17.963904 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:17.963870    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x"
Jul 17 10:04:17.964375 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:17.963897    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x" event={"ID":"c5e8923d-9adf-4a7e-b1ce-bded023d6b3c","Type":"ContainerDied","Data":"cb6bed70e9ed8bbfdeca57a086dd3c178ae01180ade5b861e602aaaa3599a9a0"}
Jul 17 10:04:17.964375 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:17.963978    2574 scope.go:117] "RemoveContainer" containerID="a6a8eee99237b8d1ec15df3bfd834af32a7e7c8ecfdff144ef17b99d69dc6e7f"
Jul 17 10:04:17.966130 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:17.966095    2574 generic.go:356] "Generic (PLEG): container finished" podID="d37bb2e2-ffd7-4004-876d-f209975bae87" containerID="a7d87e835a09088b62e7496c4b5872a1db575cbc0132dd4435a973c9851ad463" exitCode=7
Jul 17 10:04:17.966259 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:17.966171    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-89knf" event={"ID":"d37bb2e2-ffd7-4004-876d-f209975bae87","Type":"ContainerDied","Data":"a7d87e835a09088b62e7496c4b5872a1db575cbc0132dd4435a973c9851ad463"}
Jul 17 10:04:17.966373 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:17.966358    2574 scope.go:117] "RemoveContainer" containerID="a7d87e835a09088b62e7496c4b5872a1db575cbc0132dd4435a973c9851ad463"
Jul 17 10:04:17.966581 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:04:17.966563    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-89knf_odh-ai-gateway-infra(d37bb2e2-ffd7-4004-876d-f209975bae87)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-89knf" podUID="d37bb2e2-ffd7-4004-876d-f209975bae87"
Jul 17 10:04:17.973453 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:17.973430    2574 scope.go:117] "RemoveContainer" containerID="77fb99600eb96f6c71aac109d30f0f9cdeb64c4635e88c4e116ce367c8ad9611"
Jul 17 10:04:17.999962 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:17.999935    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x"]
Jul 17 10:04:18.004362 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:18.004338    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-hmk5x"]
Jul 17 10:04:18.897492 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:18.897459    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5e8923d-9adf-4a7e-b1ce-bded023d6b3c" path="/var/lib/kubelet/pods/c5e8923d-9adf-4a7e-b1ce-bded023d6b3c/volumes"
Jul 17 10:04:18.972664 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:18.972632    2574 scope.go:117] "RemoveContainer" containerID="a7d87e835a09088b62e7496c4b5872a1db575cbc0132dd4435a973c9851ad463"
Jul 17 10:04:18.973127 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:04:18.972894    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-89knf_odh-ai-gateway-infra(d37bb2e2-ffd7-4004-876d-f209975bae87)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-89knf" podUID="d37bb2e2-ffd7-4004-876d-f209975bae87"
Jul 17 10:04:27.999342 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:27.999307    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5"]
Jul 17 10:04:27.999813 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:27.999790    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5e8923d-9adf-4a7e-b1ce-bded023d6b3c" containerName="revoke-keys"
Jul 17 10:04:27.999813 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:27.999803    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5e8923d-9adf-4a7e-b1ce-bded023d6b3c" containerName="revoke-keys"
Jul 17 10:04:27.999889 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:27.999820    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5e8923d-9adf-4a7e-b1ce-bded023d6b3c" containerName="revoke-keys"
Jul 17 10:04:27.999889 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:27.999826    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5e8923d-9adf-4a7e-b1ce-bded023d6b3c" containerName="revoke-keys"
Jul 17 10:04:27.999954 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:27.999899    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="16442b8f-3ab3-4ca5-972b-16668d8640c1" containerName="revoke-keys"
Jul 17 10:04:27.999954 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:27.999907    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5e8923d-9adf-4a7e-b1ce-bded023d6b3c" containerName="revoke-keys"
Jul 17 10:04:27.999954 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:27.999914    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5e8923d-9adf-4a7e-b1ce-bded023d6b3c" containerName="revoke-keys"
Jul 17 10:04:28.004437 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:28.004420    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5"
Jul 17 10:04:28.009784 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:28.009717    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5"]
Jul 17 10:04:28.128930 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:28.128897    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf1964b8-809c-43c4-8bbf-4c734866913e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5\" (UID: \"bf1964b8-809c-43c4-8bbf-4c734866913e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5"
Jul 17 10:04:28.230487 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:28.230441    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf1964b8-809c-43c4-8bbf-4c734866913e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5\" (UID: \"bf1964b8-809c-43c4-8bbf-4c734866913e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5"
Jul 17 10:04:28.231072 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:28.231049    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf1964b8-809c-43c4-8bbf-4c734866913e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5\" (UID: \"bf1964b8-809c-43c4-8bbf-4c734866913e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5"
Jul 17 10:04:28.318797 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:28.318719    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5"
Jul 17 10:04:28.462584 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:28.462559    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5"]
Jul 17 10:04:28.465582 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:04:28.465554    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf1964b8_809c_43c4_8bbf_4c734866913e.slice/crio-f5201999f19e32f537c5219a8128c9127337ac0b681c2d4d9a33a5a48a77ade6 WatchSource:0}: Error finding container f5201999f19e32f537c5219a8128c9127337ac0b681c2d4d9a33a5a48a77ade6: Status 404 returned error can't find the container with id f5201999f19e32f537c5219a8128c9127337ac0b681c2d4d9a33a5a48a77ade6
Jul 17 10:04:29.036858 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:29.036767    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5" event={"ID":"bf1964b8-809c-43c4-8bbf-4c734866913e","Type":"ContainerStarted","Data":"3c72d6fa71a99e558fcf421ab2f5755d216aca070ffe6ef228177a9f6dbba6b3"}
Jul 17 10:04:29.036858 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:29.036845    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5" event={"ID":"bf1964b8-809c-43c4-8bbf-4c734866913e","Type":"ContainerStarted","Data":"f5201999f19e32f537c5219a8128c9127337ac0b681c2d4d9a33a5a48a77ade6"}
Jul 17 10:04:29.053828 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:29.053778    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5" podStartSLOduration=2.05375594 podStartE2EDuration="2.05375594s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:04:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:04:29.052490884 +0000 UTC m=+2266.635129271" watchObservedRunningTime="2026-07-17 10:04:29.05375594 +0000 UTC m=+2266.636394315"
Jul 17 10:04:30.043476 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:30.043438    2574 generic.go:356] "Generic (PLEG): container finished" podID="bf1964b8-809c-43c4-8bbf-4c734866913e" containerID="3c72d6fa71a99e558fcf421ab2f5755d216aca070ffe6ef228177a9f6dbba6b3" exitCode=6
Jul 17 10:04:30.043966 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:30.043527    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5" event={"ID":"bf1964b8-809c-43c4-8bbf-4c734866913e","Type":"ContainerDied","Data":"3c72d6fa71a99e558fcf421ab2f5755d216aca070ffe6ef228177a9f6dbba6b3"}
Jul 17 10:04:30.043966 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:30.043749    2574 scope.go:117] "RemoveContainer" containerID="3c72d6fa71a99e558fcf421ab2f5755d216aca070ffe6ef228177a9f6dbba6b3"
Jul 17 10:04:31.051350 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:31.051263    2574 generic.go:356] "Generic (PLEG): container finished" podID="bf1964b8-809c-43c4-8bbf-4c734866913e" containerID="949ed59803a3e73b5695ef0426b0d569404469cd52d6dad55e16cb64607806ad" exitCode=6
Jul 17 10:04:31.051350 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:31.051306    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5" event={"ID":"bf1964b8-809c-43c4-8bbf-4c734866913e","Type":"ContainerDied","Data":"949ed59803a3e73b5695ef0426b0d569404469cd52d6dad55e16cb64607806ad"}
Jul 17 10:04:31.051783 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:31.051362    2574 scope.go:117] "RemoveContainer" containerID="3c72d6fa71a99e558fcf421ab2f5755d216aca070ffe6ef228177a9f6dbba6b3"
Jul 17 10:04:31.051783 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:31.051509    2574 scope.go:117] "RemoveContainer" containerID="949ed59803a3e73b5695ef0426b0d569404469cd52d6dad55e16cb64607806ad"
Jul 17 10:04:31.051878 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:04:31.051805    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5_odh-ai-gateway-infra(bf1964b8-809c-43c4-8bbf-4c734866913e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5" podUID="bf1964b8-809c-43c4-8bbf-4c734866913e"
Jul 17 10:04:31.891615 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:31.891588    2574 scope.go:117] "RemoveContainer" containerID="a7d87e835a09088b62e7496c4b5872a1db575cbc0132dd4435a973c9851ad463"
Jul 17 10:04:32.059065 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:32.059040    2574 scope.go:117] "RemoveContainer" containerID="949ed59803a3e73b5695ef0426b0d569404469cd52d6dad55e16cb64607806ad"
Jul 17 10:04:32.059541 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:04:32.059364    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5_odh-ai-gateway-infra(bf1964b8-809c-43c4-8bbf-4c734866913e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5" podUID="bf1964b8-809c-43c4-8bbf-4c734866913e"
Jul 17 10:04:33.066293 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:33.066254    2574 generic.go:356] "Generic (PLEG): container finished" podID="d37bb2e2-ffd7-4004-876d-f209975bae87" containerID="ab206d2447d839945468f3a440bdaa16c98dd87ec0549943de4f9f72716f36e3" exitCode=7
Jul 17 10:04:33.066293 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:33.066282    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-89knf" event={"ID":"d37bb2e2-ffd7-4004-876d-f209975bae87","Type":"ContainerDied","Data":"ab206d2447d839945468f3a440bdaa16c98dd87ec0549943de4f9f72716f36e3"}
Jul 17 10:04:33.066791 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:33.066345    2574 scope.go:117] "RemoveContainer" containerID="a7d87e835a09088b62e7496c4b5872a1db575cbc0132dd4435a973c9851ad463"
Jul 17 10:04:33.066791 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:33.066625    2574 scope.go:117] "RemoveContainer" containerID="ab206d2447d839945468f3a440bdaa16c98dd87ec0549943de4f9f72716f36e3"
Jul 17 10:04:33.066875 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:04:33.066849    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-89knf_odh-ai-gateway-infra(d37bb2e2-ffd7-4004-876d-f209975bae87)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-89knf" podUID="d37bb2e2-ffd7-4004-876d-f209975bae87"
Jul 17 10:04:34.094260 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:34.094210    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-89knf"]
Jul 17 10:04:34.236226 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:34.236202    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-89knf"
Jul 17 10:04:34.288704 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:34.288669    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d37bb2e2-ffd7-4004-876d-f209975bae87-maas-api-service-ca\") pod \"d37bb2e2-ffd7-4004-876d-f209975bae87\" (UID: \"d37bb2e2-ffd7-4004-876d-f209975bae87\") "
Jul 17 10:04:34.289054 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:34.289030    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d37bb2e2-ffd7-4004-876d-f209975bae87-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d37bb2e2-ffd7-4004-876d-f209975bae87" (UID: "d37bb2e2-ffd7-4004-876d-f209975bae87"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:04:34.389775 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:34.389676    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d37bb2e2-ffd7-4004-876d-f209975bae87-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:04:35.080139 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:35.080112    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-89knf"
Jul 17 10:04:35.080322 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:35.080144    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-89knf" event={"ID":"d37bb2e2-ffd7-4004-876d-f209975bae87","Type":"ContainerDied","Data":"0bf5e7ec81f50056b6833ccf7056fe62d8299f3abe64a67f7f1a23651e1cac80"}
Jul 17 10:04:35.080322 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:35.080212    2574 scope.go:117] "RemoveContainer" containerID="ab206d2447d839945468f3a440bdaa16c98dd87ec0549943de4f9f72716f36e3"
Jul 17 10:04:35.096885 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:35.096863    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-89knf"]
Jul 17 10:04:35.098902 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:35.098883    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-89knf"]
Jul 17 10:04:35.896317 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:35.896274    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-29247"]
Jul 17 10:04:35.896814 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:35.896793    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d37bb2e2-ffd7-4004-876d-f209975bae87" containerName="revoke-keys"
Jul 17 10:04:35.896917 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:35.896817    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d37bb2e2-ffd7-4004-876d-f209975bae87" containerName="revoke-keys"
Jul 17 10:04:35.896917 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:35.896859    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5e8923d-9adf-4a7e-b1ce-bded023d6b3c" containerName="revoke-keys"
Jul 17 10:04:35.896917 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:35.896869    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5e8923d-9adf-4a7e-b1ce-bded023d6b3c" containerName="revoke-keys"
Jul 17 10:04:35.896917 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:35.896890    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d37bb2e2-ffd7-4004-876d-f209975bae87" containerName="revoke-keys"
Jul 17 10:04:35.896917 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:35.896898    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d37bb2e2-ffd7-4004-876d-f209975bae87" containerName="revoke-keys"
Jul 17 10:04:35.897175 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:35.896994    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d37bb2e2-ffd7-4004-876d-f209975bae87" containerName="revoke-keys"
Jul 17 10:04:35.897175 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:35.897013    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d37bb2e2-ffd7-4004-876d-f209975bae87" containerName="revoke-keys"
Jul 17 10:04:35.897175 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:35.897023    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5e8923d-9adf-4a7e-b1ce-bded023d6b3c" containerName="revoke-keys"
Jul 17 10:04:35.901503 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:35.901479    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-29247"
Jul 17 10:04:35.908179 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:35.907882    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-29247"]
Jul 17 10:04:36.000865 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:36.000817    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a7faf5c-384f-411e-b519-f58a06c1f845-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-29247\" (UID: \"0a7faf5c-384f-411e-b519-f58a06c1f845\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-29247"
Jul 17 10:04:36.101506 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:36.101470    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a7faf5c-384f-411e-b519-f58a06c1f845-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-29247\" (UID: \"0a7faf5c-384f-411e-b519-f58a06c1f845\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-29247"
Jul 17 10:04:36.102098 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:36.102074    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a7faf5c-384f-411e-b519-f58a06c1f845-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-29247\" (UID: \"0a7faf5c-384f-411e-b519-f58a06c1f845\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-29247"
Jul 17 10:04:36.215028 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:36.214988    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-29247"
Jul 17 10:04:36.564354 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:36.564325    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-29247"]
Jul 17 10:04:36.566567 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:04:36.566525    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a7faf5c_384f_411e_b519_f58a06c1f845.slice/crio-39ad79acb5762b442ddfb463d951acc37f78647a07aac5eb1331ca5a789f30a9 WatchSource:0}: Error finding container 39ad79acb5762b442ddfb463d951acc37f78647a07aac5eb1331ca5a789f30a9: Status 404 returned error can't find the container with id 39ad79acb5762b442ddfb463d951acc37f78647a07aac5eb1331ca5a789f30a9
Jul 17 10:04:36.898286 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:36.898228    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d37bb2e2-ffd7-4004-876d-f209975bae87" path="/var/lib/kubelet/pods/d37bb2e2-ffd7-4004-876d-f209975bae87/volumes"
Jul 17 10:04:37.095357 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:37.095268    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-29247" event={"ID":"0a7faf5c-384f-411e-b519-f58a06c1f845","Type":"ContainerStarted","Data":"4da27d6e61b20dba7727bb50614201d17d6df07f19185e34b5bebf3b678067a1"}
Jul 17 10:04:37.095357 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:37.095318    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-29247" event={"ID":"0a7faf5c-384f-411e-b519-f58a06c1f845","Type":"ContainerStarted","Data":"39ad79acb5762b442ddfb463d951acc37f78647a07aac5eb1331ca5a789f30a9"}
Jul 17 10:04:37.113448 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:37.113399    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-29247" podStartSLOduration=2.113383329 podStartE2EDuration="2.113383329s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:04:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:04:37.111625205 +0000 UTC m=+2274.694263580" watchObservedRunningTime="2026-07-17 10:04:37.113383329 +0000 UTC m=+2274.696021719"
Jul 17 10:04:38.102000 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:38.101967    2574 generic.go:356] "Generic (PLEG): container finished" podID="0a7faf5c-384f-411e-b519-f58a06c1f845" containerID="4da27d6e61b20dba7727bb50614201d17d6df07f19185e34b5bebf3b678067a1" exitCode=6
Jul 17 10:04:38.102186 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:38.102049    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-29247" event={"ID":"0a7faf5c-384f-411e-b519-f58a06c1f845","Type":"ContainerDied","Data":"4da27d6e61b20dba7727bb50614201d17d6df07f19185e34b5bebf3b678067a1"}
Jul 17 10:04:38.102233 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:38.102212    2574 scope.go:117] "RemoveContainer" containerID="4da27d6e61b20dba7727bb50614201d17d6df07f19185e34b5bebf3b678067a1"
Jul 17 10:04:39.113126 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:39.113033    2574 generic.go:356] "Generic (PLEG): container finished" podID="0a7faf5c-384f-411e-b519-f58a06c1f845" containerID="81fefb962cdb87bb9ca99fb07c5e03c6e6b74ed8650155a1bf5355d98e6996c6" exitCode=6
Jul 17 10:04:39.113126 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:39.113089    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-29247" event={"ID":"0a7faf5c-384f-411e-b519-f58a06c1f845","Type":"ContainerDied","Data":"81fefb962cdb87bb9ca99fb07c5e03c6e6b74ed8650155a1bf5355d98e6996c6"}
Jul 17 10:04:39.113639 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:39.113145    2574 scope.go:117] "RemoveContainer" containerID="4da27d6e61b20dba7727bb50614201d17d6df07f19185e34b5bebf3b678067a1"
Jul 17 10:04:39.113639 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:39.113363    2574 scope.go:117] "RemoveContainer" containerID="81fefb962cdb87bb9ca99fb07c5e03c6e6b74ed8650155a1bf5355d98e6996c6"
Jul 17 10:04:39.113639 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:04:39.113610    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-29247_odh-ai-gateway-infra(0a7faf5c-384f-411e-b519-f58a06c1f845)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-29247" podUID="0a7faf5c-384f-411e-b519-f58a06c1f845"
Jul 17 10:04:40.122434 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:40.122402    2574 scope.go:117] "RemoveContainer" containerID="81fefb962cdb87bb9ca99fb07c5e03c6e6b74ed8650155a1bf5355d98e6996c6"
Jul 17 10:04:40.122913 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:04:40.122659    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-29247_odh-ai-gateway-infra(0a7faf5c-384f-411e-b519-f58a06c1f845)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-29247" podUID="0a7faf5c-384f-411e-b519-f58a06c1f845"
Jul 17 10:04:45.891710 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:45.891678    2574 scope.go:117] "RemoveContainer" containerID="949ed59803a3e73b5695ef0426b0d569404469cd52d6dad55e16cb64607806ad"
Jul 17 10:04:47.162500 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:47.162467    2574 generic.go:356] "Generic (PLEG): container finished" podID="bf1964b8-809c-43c4-8bbf-4c734866913e" containerID="846c174ce33fbd5ea23f643fb03e768ec8f31f08a56f54f3e8b11b72099d7115" exitCode=6
Jul 17 10:04:47.162876 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:47.162543    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5" event={"ID":"bf1964b8-809c-43c4-8bbf-4c734866913e","Type":"ContainerDied","Data":"846c174ce33fbd5ea23f643fb03e768ec8f31f08a56f54f3e8b11b72099d7115"}
Jul 17 10:04:47.162876 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:47.162604    2574 scope.go:117] "RemoveContainer" containerID="949ed59803a3e73b5695ef0426b0d569404469cd52d6dad55e16cb64607806ad"
Jul 17 10:04:47.163166 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:47.162928    2574 scope.go:117] "RemoveContainer" containerID="846c174ce33fbd5ea23f643fb03e768ec8f31f08a56f54f3e8b11b72099d7115"
Jul 17 10:04:47.163234 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:04:47.163203    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5_odh-ai-gateway-infra(bf1964b8-809c-43c4-8bbf-4c734866913e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5" podUID="bf1964b8-809c-43c4-8bbf-4c734866913e"
Jul 17 10:04:48.188475 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:48.188444    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5"]
Jul 17 10:04:48.334419 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:48.334397    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5"
Jul 17 10:04:48.414091 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:48.414056    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf1964b8-809c-43c4-8bbf-4c734866913e-maas-api-service-ca\") pod \"bf1964b8-809c-43c4-8bbf-4c734866913e\" (UID: \"bf1964b8-809c-43c4-8bbf-4c734866913e\") "
Jul 17 10:04:48.414418 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:48.414396    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf1964b8-809c-43c4-8bbf-4c734866913e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bf1964b8-809c-43c4-8bbf-4c734866913e" (UID: "bf1964b8-809c-43c4-8bbf-4c734866913e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:04:48.515081 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:48.515012    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf1964b8-809c-43c4-8bbf-4c734866913e-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:04:49.176295 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:49.176272    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5"
Jul 17 10:04:49.176462 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:49.176262    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5" event={"ID":"bf1964b8-809c-43c4-8bbf-4c734866913e","Type":"ContainerDied","Data":"f5201999f19e32f537c5219a8128c9127337ac0b681c2d4d9a33a5a48a77ade6"}
Jul 17 10:04:49.176462 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:49.176401    2574 scope.go:117] "RemoveContainer" containerID="846c174ce33fbd5ea23f643fb03e768ec8f31f08a56f54f3e8b11b72099d7115"
Jul 17 10:04:49.197261 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:49.197222    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5"]
Jul 17 10:04:49.204589 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:49.204565    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-xgrw5"]
Jul 17 10:04:50.898376 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:50.898343    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf1964b8-809c-43c4-8bbf-4c734866913e" path="/var/lib/kubelet/pods/bf1964b8-809c-43c4-8bbf-4c734866913e/volumes"
Jul 17 10:04:52.894723 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:52.894687    2574 scope.go:117] "RemoveContainer" containerID="81fefb962cdb87bb9ca99fb07c5e03c6e6b74ed8650155a1bf5355d98e6996c6"
Jul 17 10:04:53.203491 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:53.203444    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-29247" event={"ID":"0a7faf5c-384f-411e-b519-f58a06c1f845","Type":"ContainerStarted","Data":"7e76825315c435175c7f5168bb7138711e084d1abdf750c5f78dd8038ca1c059"}
Jul 17 10:04:54.210616 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:54.210585    2574 generic.go:356] "Generic (PLEG): container finished" podID="0a7faf5c-384f-411e-b519-f58a06c1f845" containerID="7e76825315c435175c7f5168bb7138711e084d1abdf750c5f78dd8038ca1c059" exitCode=6
Jul 17 10:04:54.211048 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:54.210633    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-29247" event={"ID":"0a7faf5c-384f-411e-b519-f58a06c1f845","Type":"ContainerDied","Data":"7e76825315c435175c7f5168bb7138711e084d1abdf750c5f78dd8038ca1c059"}
Jul 17 10:04:54.211048 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:54.210680    2574 scope.go:117] "RemoveContainer" containerID="81fefb962cdb87bb9ca99fb07c5e03c6e6b74ed8650155a1bf5355d98e6996c6"
Jul 17 10:04:54.211048 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:54.210990    2574 scope.go:117] "RemoveContainer" containerID="7e76825315c435175c7f5168bb7138711e084d1abdf750c5f78dd8038ca1c059"
Jul 17 10:04:54.211302 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:04:54.211279    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-29247_odh-ai-gateway-infra(0a7faf5c-384f-411e-b519-f58a06c1f845)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-29247" podUID="0a7faf5c-384f-411e-b519-f58a06c1f845"
Jul 17 10:04:54.240304 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:54.240273    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-29247"]
Jul 17 10:04:55.357137 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:55.357113    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-29247"
Jul 17 10:04:55.483229 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:55.483137    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a7faf5c-384f-411e-b519-f58a06c1f845-maas-api-service-ca\") pod \"0a7faf5c-384f-411e-b519-f58a06c1f845\" (UID: \"0a7faf5c-384f-411e-b519-f58a06c1f845\") "
Jul 17 10:04:55.483532 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:55.483506    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a7faf5c-384f-411e-b519-f58a06c1f845-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0a7faf5c-384f-411e-b519-f58a06c1f845" (UID: "0a7faf5c-384f-411e-b519-f58a06c1f845"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:04:55.584101 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:55.584063    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a7faf5c-384f-411e-b519-f58a06c1f845-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:04:56.223899 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.223857    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-29247" event={"ID":"0a7faf5c-384f-411e-b519-f58a06c1f845","Type":"ContainerDied","Data":"39ad79acb5762b442ddfb463d951acc37f78647a07aac5eb1331ca5a789f30a9"}
Jul 17 10:04:56.224093 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.223918    2574 scope.go:117] "RemoveContainer" containerID="7e76825315c435175c7f5168bb7138711e084d1abdf750c5f78dd8038ca1c059"
Jul 17 10:04:56.224093 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.223873    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-29247"
Jul 17 10:04:56.253438 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.251225    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-29247"]
Jul 17 10:04:56.259638 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.259611    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-29247"]
Jul 17 10:04:56.857457 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.857421    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc"]
Jul 17 10:04:56.857959 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.857943    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a7faf5c-384f-411e-b519-f58a06c1f845" containerName="revoke-keys"
Jul 17 10:04:56.858005 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.857961    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a7faf5c-384f-411e-b519-f58a06c1f845" containerName="revoke-keys"
Jul 17 10:04:56.858005 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.857968    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf1964b8-809c-43c4-8bbf-4c734866913e" containerName="revoke-keys"
Jul 17 10:04:56.858005 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.857973    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf1964b8-809c-43c4-8bbf-4c734866913e" containerName="revoke-keys"
Jul 17 10:04:56.858005 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.857980    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf1964b8-809c-43c4-8bbf-4c734866913e" containerName="revoke-keys"
Jul 17 10:04:56.858005 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.857988    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf1964b8-809c-43c4-8bbf-4c734866913e" containerName="revoke-keys"
Jul 17 10:04:56.858159 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.858006    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d37bb2e2-ffd7-4004-876d-f209975bae87" containerName="revoke-keys"
Jul 17 10:04:56.858159 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.858012    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d37bb2e2-ffd7-4004-876d-f209975bae87" containerName="revoke-keys"
Jul 17 10:04:56.858159 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.858029    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a7faf5c-384f-411e-b519-f58a06c1f845" containerName="revoke-keys"
Jul 17 10:04:56.858159 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.858034    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a7faf5c-384f-411e-b519-f58a06c1f845" containerName="revoke-keys"
Jul 17 10:04:56.858159 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.858040    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bf1964b8-809c-43c4-8bbf-4c734866913e" containerName="revoke-keys"
Jul 17 10:04:56.858159 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.858044    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bf1964b8-809c-43c4-8bbf-4c734866913e" containerName="revoke-keys"
Jul 17 10:04:56.858159 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.858055    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a7faf5c-384f-411e-b519-f58a06c1f845" containerName="revoke-keys"
Jul 17 10:04:56.858159 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.858060    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a7faf5c-384f-411e-b519-f58a06c1f845" containerName="revoke-keys"
Jul 17 10:04:56.858159 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.858132    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf1964b8-809c-43c4-8bbf-4c734866913e" containerName="revoke-keys"
Jul 17 10:04:56.858159 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.858143    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf1964b8-809c-43c4-8bbf-4c734866913e" containerName="revoke-keys"
Jul 17 10:04:56.858159 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.858152    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a7faf5c-384f-411e-b519-f58a06c1f845" containerName="revoke-keys"
Jul 17 10:04:56.858159 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.858163    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d37bb2e2-ffd7-4004-876d-f209975bae87" containerName="revoke-keys"
Jul 17 10:04:56.858581 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.858169    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a7faf5c-384f-411e-b519-f58a06c1f845" containerName="revoke-keys"
Jul 17 10:04:56.858581 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.858174    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="bf1964b8-809c-43c4-8bbf-4c734866913e" containerName="revoke-keys"
Jul 17 10:04:56.862522 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.862505    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc"
Jul 17 10:04:56.865474 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.865445    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:04:56.870295 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.870267    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc"]
Jul 17 10:04:56.898679 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.898646    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a7faf5c-384f-411e-b519-f58a06c1f845" path="/var/lib/kubelet/pods/0a7faf5c-384f-411e-b519-f58a06c1f845/volumes"
Jul 17 10:04:56.999357 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:56.999323    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f019bdd-ac0e-4dac-a3c7-dc24f72d083d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc\" (UID: \"9f019bdd-ac0e-4dac-a3c7-dc24f72d083d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc"
Jul 17 10:04:57.100096 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:57.100063    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f019bdd-ac0e-4dac-a3c7-dc24f72d083d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc\" (UID: \"9f019bdd-ac0e-4dac-a3c7-dc24f72d083d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc"
Jul 17 10:04:57.100742 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:57.100714    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f019bdd-ac0e-4dac-a3c7-dc24f72d083d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc\" (UID: \"9f019bdd-ac0e-4dac-a3c7-dc24f72d083d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc"
Jul 17 10:04:57.174745 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:57.174716    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc"
Jul 17 10:04:57.314509 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:57.314481    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc"]
Jul 17 10:04:57.316422 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:04:57.316393    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f019bdd_ac0e_4dac_a3c7_dc24f72d083d.slice/crio-5dd46c8a159ff176ad06e87f0dce46587793b7b166e7b4c8ce315922b301af13 WatchSource:0}: Error finding container 5dd46c8a159ff176ad06e87f0dce46587793b7b166e7b4c8ce315922b301af13: Status 404 returned error can't find the container with id 5dd46c8a159ff176ad06e87f0dce46587793b7b166e7b4c8ce315922b301af13
Jul 17 10:04:58.240612 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:58.240578    2574 generic.go:356] "Generic (PLEG): container finished" podID="9f019bdd-ac0e-4dac-a3c7-dc24f72d083d" containerID="61bd7505e2a16f9678382796808d6cbcf2af204c06f926a99536f08a4a733416" exitCode=6
Jul 17 10:04:58.241049 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:58.240625    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc" event={"ID":"9f019bdd-ac0e-4dac-a3c7-dc24f72d083d","Type":"ContainerDied","Data":"61bd7505e2a16f9678382796808d6cbcf2af204c06f926a99536f08a4a733416"}
Jul 17 10:04:58.241049 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:58.240683    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc" event={"ID":"9f019bdd-ac0e-4dac-a3c7-dc24f72d083d","Type":"ContainerStarted","Data":"5dd46c8a159ff176ad06e87f0dce46587793b7b166e7b4c8ce315922b301af13"}
Jul 17 10:04:58.241049 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:58.240885    2574 scope.go:117] "RemoveContainer" containerID="61bd7505e2a16f9678382796808d6cbcf2af204c06f926a99536f08a4a733416"
Jul 17 10:04:59.247630 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:59.247597    2574 generic.go:356] "Generic (PLEG): container finished" podID="9f019bdd-ac0e-4dac-a3c7-dc24f72d083d" containerID="4802f9041b2720f6460ea0e12c2af957d8b185b09281f394683cf295ceeb0c38" exitCode=6
Jul 17 10:04:59.248101 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:59.247691    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc" event={"ID":"9f019bdd-ac0e-4dac-a3c7-dc24f72d083d","Type":"ContainerDied","Data":"4802f9041b2720f6460ea0e12c2af957d8b185b09281f394683cf295ceeb0c38"}
Jul 17 10:04:59.248101 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:59.247749    2574 scope.go:117] "RemoveContainer" containerID="61bd7505e2a16f9678382796808d6cbcf2af204c06f926a99536f08a4a733416"
Jul 17 10:04:59.248101 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:04:59.247879    2574 scope.go:117] "RemoveContainer" containerID="4802f9041b2720f6460ea0e12c2af957d8b185b09281f394683cf295ceeb0c38"
Jul 17 10:04:59.248234 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:04:59.248105    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc_odh-ai-gateway-infra(9f019bdd-ac0e-4dac-a3c7-dc24f72d083d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc" podUID="9f019bdd-ac0e-4dac-a3c7-dc24f72d083d"
Jul 17 10:05:00.262512 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:00.262486    2574 scope.go:117] "RemoveContainer" containerID="4802f9041b2720f6460ea0e12c2af957d8b185b09281f394683cf295ceeb0c38"
Jul 17 10:05:00.262902 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:05:00.262671    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc_odh-ai-gateway-infra(9f019bdd-ac0e-4dac-a3c7-dc24f72d083d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc" podUID="9f019bdd-ac0e-4dac-a3c7-dc24f72d083d"
Jul 17 10:05:05.324623 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:05.324586    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-79p5x"]
Jul 17 10:05:05.325103 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:05.325079    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a7faf5c-384f-411e-b519-f58a06c1f845" containerName="revoke-keys"
Jul 17 10:05:05.329571 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:05.329553    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-79p5x"
Jul 17 10:05:05.345849 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:05.345822    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-79p5x"]
Jul 17 10:05:05.488096 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:05.488058    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f4a870f-e7d9-4729-bfe9-c147c732cec1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-79p5x\" (UID: \"1f4a870f-e7d9-4729-bfe9-c147c732cec1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-79p5x"
Jul 17 10:05:05.588841 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:05.588761    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f4a870f-e7d9-4729-bfe9-c147c732cec1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-79p5x\" (UID: \"1f4a870f-e7d9-4729-bfe9-c147c732cec1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-79p5x"
Jul 17 10:05:05.589419 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:05.589401    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f4a870f-e7d9-4729-bfe9-c147c732cec1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-79p5x\" (UID: \"1f4a870f-e7d9-4729-bfe9-c147c732cec1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-79p5x"
Jul 17 10:05:05.640029 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:05.639987    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-79p5x"
Jul 17 10:05:05.786146 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:05.786115    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-79p5x"]
Jul 17 10:05:05.788385 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:05:05.788340    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f4a870f_e7d9_4729_bfe9_c147c732cec1.slice/crio-992020c888c29b8c48bf9446e625d44fd43905a99eb5bb5ce54943aaff7f5903 WatchSource:0}: Error finding container 992020c888c29b8c48bf9446e625d44fd43905a99eb5bb5ce54943aaff7f5903: Status 404 returned error can't find the container with id 992020c888c29b8c48bf9446e625d44fd43905a99eb5bb5ce54943aaff7f5903
Jul 17 10:05:06.294311 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:06.294194    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-79p5x" event={"ID":"1f4a870f-e7d9-4729-bfe9-c147c732cec1","Type":"ContainerStarted","Data":"bb7ad2cfee61dac19fa007fb23da9f1bd858f9cdfbb77fde3cc930446404ff4e"}
Jul 17 10:05:06.294311 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:06.294271    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-79p5x" event={"ID":"1f4a870f-e7d9-4729-bfe9-c147c732cec1","Type":"ContainerStarted","Data":"992020c888c29b8c48bf9446e625d44fd43905a99eb5bb5ce54943aaff7f5903"}
Jul 17 10:05:06.315472 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:06.315426    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-79p5x" podStartSLOduration=1.3154104819999999 podStartE2EDuration="1.315410482s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:05:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:05:06.314595073 +0000 UTC m=+2303.897233449" watchObservedRunningTime="2026-07-17 10:05:06.315410482 +0000 UTC m=+2303.898048858"
Jul 17 10:05:07.301616 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:07.301578    2574 generic.go:356] "Generic (PLEG): container finished" podID="1f4a870f-e7d9-4729-bfe9-c147c732cec1" containerID="bb7ad2cfee61dac19fa007fb23da9f1bd858f9cdfbb77fde3cc930446404ff4e" exitCode=7
Jul 17 10:05:07.302128 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:07.301625    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-79p5x" event={"ID":"1f4a870f-e7d9-4729-bfe9-c147c732cec1","Type":"ContainerDied","Data":"bb7ad2cfee61dac19fa007fb23da9f1bd858f9cdfbb77fde3cc930446404ff4e"}
Jul 17 10:05:07.302128 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:07.301948    2574 scope.go:117] "RemoveContainer" containerID="bb7ad2cfee61dac19fa007fb23da9f1bd858f9cdfbb77fde3cc930446404ff4e"
Jul 17 10:05:08.309077 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:08.309039    2574 generic.go:356] "Generic (PLEG): container finished" podID="1f4a870f-e7d9-4729-bfe9-c147c732cec1" containerID="43dc7a49769478f93f01de56f0433a254e26dbd7c5f48ec5b98ede5357bf9aa0" exitCode=7
Jul 17 10:05:08.309506 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:08.309129    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-79p5x" event={"ID":"1f4a870f-e7d9-4729-bfe9-c147c732cec1","Type":"ContainerDied","Data":"43dc7a49769478f93f01de56f0433a254e26dbd7c5f48ec5b98ede5357bf9aa0"}
Jul 17 10:05:08.309506 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:08.309196    2574 scope.go:117] "RemoveContainer" containerID="bb7ad2cfee61dac19fa007fb23da9f1bd858f9cdfbb77fde3cc930446404ff4e"
Jul 17 10:05:08.309506 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:08.309446    2574 scope.go:117] "RemoveContainer" containerID="43dc7a49769478f93f01de56f0433a254e26dbd7c5f48ec5b98ede5357bf9aa0"
Jul 17 10:05:08.309718 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:05:08.309700    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-79p5x_odh-ai-gateway-infra(1f4a870f-e7d9-4729-bfe9-c147c732cec1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-79p5x" podUID="1f4a870f-e7d9-4729-bfe9-c147c732cec1"
Jul 17 10:05:09.316407 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:09.316379    2574 scope.go:117] "RemoveContainer" containerID="43dc7a49769478f93f01de56f0433a254e26dbd7c5f48ec5b98ede5357bf9aa0"
Jul 17 10:05:09.316799 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:05:09.316603    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-79p5x_odh-ai-gateway-infra(1f4a870f-e7d9-4729-bfe9-c147c732cec1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-79p5x" podUID="1f4a870f-e7d9-4729-bfe9-c147c732cec1"
Jul 17 10:05:12.894285 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:12.894235    2574 scope.go:117] "RemoveContainer" containerID="4802f9041b2720f6460ea0e12c2af957d8b185b09281f394683cf295ceeb0c38"
Jul 17 10:05:13.345561 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:13.345520    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc" event={"ID":"9f019bdd-ac0e-4dac-a3c7-dc24f72d083d","Type":"ContainerStarted","Data":"c79c14fb7c9017bc86d388f5a154a5bc2dfd9f342b6b6a409fe683bc5341128c"}
Jul 17 10:05:13.365042 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:13.364994    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc" podStartSLOduration=17.364980291 podStartE2EDuration="17.364980291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:04:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:05:13.364188115 +0000 UTC m=+2310.946826512" watchObservedRunningTime="2026-07-17 10:05:13.364980291 +0000 UTC m=+2310.947618698"
Jul 17 10:05:14.352573 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:14.352534    2574 generic.go:356] "Generic (PLEG): container finished" podID="9f019bdd-ac0e-4dac-a3c7-dc24f72d083d" containerID="c79c14fb7c9017bc86d388f5a154a5bc2dfd9f342b6b6a409fe683bc5341128c" exitCode=6
Jul 17 10:05:14.353004 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:14.352614    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc" event={"ID":"9f019bdd-ac0e-4dac-a3c7-dc24f72d083d","Type":"ContainerDied","Data":"c79c14fb7c9017bc86d388f5a154a5bc2dfd9f342b6b6a409fe683bc5341128c"}
Jul 17 10:05:14.353004 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:14.352677    2574 scope.go:117] "RemoveContainer" containerID="4802f9041b2720f6460ea0e12c2af957d8b185b09281f394683cf295ceeb0c38"
Jul 17 10:05:14.353004 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:14.352988    2574 scope.go:117] "RemoveContainer" containerID="c79c14fb7c9017bc86d388f5a154a5bc2dfd9f342b6b6a409fe683bc5341128c"
Jul 17 10:05:14.353327 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:05:14.353304    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc_odh-ai-gateway-infra(9f019bdd-ac0e-4dac-a3c7-dc24f72d083d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc" podUID="9f019bdd-ac0e-4dac-a3c7-dc24f72d083d"
Jul 17 10:05:14.375280 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:14.375252    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc"]
Jul 17 10:05:15.505751 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:15.505719    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc"
Jul 17 10:05:15.687492 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:15.687456    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f019bdd-ac0e-4dac-a3c7-dc24f72d083d-maas-api-service-ca\") pod \"9f019bdd-ac0e-4dac-a3c7-dc24f72d083d\" (UID: \"9f019bdd-ac0e-4dac-a3c7-dc24f72d083d\") "
Jul 17 10:05:15.687791 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:15.687769    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f019bdd-ac0e-4dac-a3c7-dc24f72d083d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9f019bdd-ac0e-4dac-a3c7-dc24f72d083d" (UID: "9f019bdd-ac0e-4dac-a3c7-dc24f72d083d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:05:15.788416 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:15.788379    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f019bdd-ac0e-4dac-a3c7-dc24f72d083d-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:05:16.366946 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:16.366910    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc" event={"ID":"9f019bdd-ac0e-4dac-a3c7-dc24f72d083d","Type":"ContainerDied","Data":"5dd46c8a159ff176ad06e87f0dce46587793b7b166e7b4c8ce315922b301af13"}
Jul 17 10:05:16.366946 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:16.366932    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc"
Jul 17 10:05:16.367228 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:16.366970    2574 scope.go:117] "RemoveContainer" containerID="c79c14fb7c9017bc86d388f5a154a5bc2dfd9f342b6b6a409fe683bc5341128c"
Jul 17 10:05:16.391198 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:16.391176    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc"]
Jul 17 10:05:16.395185 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:16.395163    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-bgxkc"]
Jul 17 10:05:16.897831 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:16.897787    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f019bdd-ac0e-4dac-a3c7-dc24f72d083d" path="/var/lib/kubelet/pods/9f019bdd-ac0e-4dac-a3c7-dc24f72d083d/volumes"
Jul 17 10:05:21.891369 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:21.891335    2574 scope.go:117] "RemoveContainer" containerID="43dc7a49769478f93f01de56f0433a254e26dbd7c5f48ec5b98ede5357bf9aa0"
Jul 17 10:05:22.401666 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:22.401632    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-79p5x" event={"ID":"1f4a870f-e7d9-4729-bfe9-c147c732cec1","Type":"ContainerStarted","Data":"ce6426d152a9e62eb798e784e0465cc68a563fbdd3fc451dcdde0f70f79804b2"}
Jul 17 10:05:23.408847 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:23.408811    2574 generic.go:356] "Generic (PLEG): container finished" podID="1f4a870f-e7d9-4729-bfe9-c147c732cec1" containerID="ce6426d152a9e62eb798e784e0465cc68a563fbdd3fc451dcdde0f70f79804b2" exitCode=7
Jul 17 10:05:23.409306 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:23.408889    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-79p5x" event={"ID":"1f4a870f-e7d9-4729-bfe9-c147c732cec1","Type":"ContainerDied","Data":"ce6426d152a9e62eb798e784e0465cc68a563fbdd3fc451dcdde0f70f79804b2"}
Jul 17 10:05:23.409306 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:23.408962    2574 scope.go:117] "RemoveContainer" containerID="43dc7a49769478f93f01de56f0433a254e26dbd7c5f48ec5b98ede5357bf9aa0"
Jul 17 10:05:23.409306 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:23.409233    2574 scope.go:117] "RemoveContainer" containerID="ce6426d152a9e62eb798e784e0465cc68a563fbdd3fc451dcdde0f70f79804b2"
Jul 17 10:05:23.409509 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:05:23.409482    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-79p5x_odh-ai-gateway-infra(1f4a870f-e7d9-4729-bfe9-c147c732cec1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-79p5x" podUID="1f4a870f-e7d9-4729-bfe9-c147c732cec1"
Jul 17 10:05:23.429496 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:23.429472    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-79p5x"]
Jul 17 10:05:24.559328 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:24.559301    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-79p5x"
Jul 17 10:05:24.565506 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:24.565483    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f4a870f-e7d9-4729-bfe9-c147c732cec1-maas-api-service-ca\") pod \"1f4a870f-e7d9-4729-bfe9-c147c732cec1\" (UID: \"1f4a870f-e7d9-4729-bfe9-c147c732cec1\") "
Jul 17 10:05:24.565828 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:24.565807    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f4a870f-e7d9-4729-bfe9-c147c732cec1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1f4a870f-e7d9-4729-bfe9-c147c732cec1" (UID: "1f4a870f-e7d9-4729-bfe9-c147c732cec1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:05:24.666267 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:24.666224    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f4a870f-e7d9-4729-bfe9-c147c732cec1-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:05:25.430299 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:25.430263    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-79p5x"
Jul 17 10:05:25.430299 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:25.430280    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-79p5x" event={"ID":"1f4a870f-e7d9-4729-bfe9-c147c732cec1","Type":"ContainerDied","Data":"992020c888c29b8c48bf9446e625d44fd43905a99eb5bb5ce54943aaff7f5903"}
Jul 17 10:05:25.430525 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:25.430384    2574 scope.go:117] "RemoveContainer" containerID="ce6426d152a9e62eb798e784e0465cc68a563fbdd3fc451dcdde0f70f79804b2"
Jul 17 10:05:25.453072 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:25.453037    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-79p5x"]
Jul 17 10:05:25.455086 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:25.455059    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-79p5x"]
Jul 17 10:05:26.898040 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:26.898008    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f4a870f-e7d9-4729-bfe9-c147c732cec1" path="/var/lib/kubelet/pods/1f4a870f-e7d9-4729-bfe9-c147c732cec1/volumes"
Jul 17 10:05:28.042296 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:28.042251    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9tbsj"]
Jul 17 10:05:28.042897 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:28.042874    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f4a870f-e7d9-4729-bfe9-c147c732cec1" containerName="revoke-keys"
Jul 17 10:05:28.042997 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:28.042900    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f4a870f-e7d9-4729-bfe9-c147c732cec1" containerName="revoke-keys"
Jul 17 10:05:28.042997 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:28.042927    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f019bdd-ac0e-4dac-a3c7-dc24f72d083d" containerName="revoke-keys"
Jul 17 10:05:28.042997 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:28.042936    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f019bdd-ac0e-4dac-a3c7-dc24f72d083d" containerName="revoke-keys"
Jul 17 10:05:28.042997 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:28.042946    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f4a870f-e7d9-4729-bfe9-c147c732cec1" containerName="revoke-keys"
Jul 17 10:05:28.042997 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:28.042955    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f4a870f-e7d9-4729-bfe9-c147c732cec1" containerName="revoke-keys"
Jul 17 10:05:28.042997 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:28.042978    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f019bdd-ac0e-4dac-a3c7-dc24f72d083d" containerName="revoke-keys"
Jul 17 10:05:28.042997 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:28.042986    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f019bdd-ac0e-4dac-a3c7-dc24f72d083d" containerName="revoke-keys"
Jul 17 10:05:28.043383 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:28.043102    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f4a870f-e7d9-4729-bfe9-c147c732cec1" containerName="revoke-keys"
Jul 17 10:05:28.043383 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:28.043115    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f019bdd-ac0e-4dac-a3c7-dc24f72d083d" containerName="revoke-keys"
Jul 17 10:05:28.043383 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:28.043127    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f4a870f-e7d9-4729-bfe9-c147c732cec1" containerName="revoke-keys"
Jul 17 10:05:28.043383 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:28.043142    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f019bdd-ac0e-4dac-a3c7-dc24f72d083d" containerName="revoke-keys"
Jul 17 10:05:28.047708 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:28.047686    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9tbsj"
Jul 17 10:05:28.050697 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:28.050676    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:05:28.052767 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:28.052743    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9tbsj"]
Jul 17 10:05:28.099069 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:28.099041    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/575bac5c-c3c4-4742-8b20-b5b6e575fb06-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-9tbsj\" (UID: \"575bac5c-c3c4-4742-8b20-b5b6e575fb06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9tbsj"
Jul 17 10:05:28.200021 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:28.199985    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/575bac5c-c3c4-4742-8b20-b5b6e575fb06-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-9tbsj\" (UID: \"575bac5c-c3c4-4742-8b20-b5b6e575fb06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9tbsj"
Jul 17 10:05:28.200708 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:28.200687    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/575bac5c-c3c4-4742-8b20-b5b6e575fb06-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-9tbsj\" (UID: \"575bac5c-c3c4-4742-8b20-b5b6e575fb06\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9tbsj"
Jul 17 10:05:28.360754 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:28.360669    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9tbsj"
Jul 17 10:05:28.502461 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:28.502437    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9tbsj"]
Jul 17 10:05:28.504722 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:05:28.504696    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod575bac5c_c3c4_4742_8b20_b5b6e575fb06.slice/crio-9622e54e38807d978a37841fd38f97246ffaf01ca1b154a396a1219a933afa17 WatchSource:0}: Error finding container 9622e54e38807d978a37841fd38f97246ffaf01ca1b154a396a1219a933afa17: Status 404 returned error can't find the container with id 9622e54e38807d978a37841fd38f97246ffaf01ca1b154a396a1219a933afa17
Jul 17 10:05:29.453891 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:29.453858    2574 generic.go:356] "Generic (PLEG): container finished" podID="575bac5c-c3c4-4742-8b20-b5b6e575fb06" containerID="f955cb43fb46797ceaed6e28fc161a3ce4d0521a391e4ab31abe6e0e729bd824" exitCode=6
Jul 17 10:05:29.454371 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:29.453926    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9tbsj" event={"ID":"575bac5c-c3c4-4742-8b20-b5b6e575fb06","Type":"ContainerDied","Data":"f955cb43fb46797ceaed6e28fc161a3ce4d0521a391e4ab31abe6e0e729bd824"}
Jul 17 10:05:29.454371 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:29.453971    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9tbsj" event={"ID":"575bac5c-c3c4-4742-8b20-b5b6e575fb06","Type":"ContainerStarted","Data":"9622e54e38807d978a37841fd38f97246ffaf01ca1b154a396a1219a933afa17"}
Jul 17 10:05:29.454371 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:29.454162    2574 scope.go:117] "RemoveContainer" containerID="f955cb43fb46797ceaed6e28fc161a3ce4d0521a391e4ab31abe6e0e729bd824"
Jul 17 10:05:30.461289 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:30.461253    2574 generic.go:356] "Generic (PLEG): container finished" podID="575bac5c-c3c4-4742-8b20-b5b6e575fb06" containerID="32b182873c2f8449e9e84ac35de46f52af6baa939ea723320e40cc4d9597eeb3" exitCode=6
Jul 17 10:05:30.461714 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:30.461346    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9tbsj" event={"ID":"575bac5c-c3c4-4742-8b20-b5b6e575fb06","Type":"ContainerDied","Data":"32b182873c2f8449e9e84ac35de46f52af6baa939ea723320e40cc4d9597eeb3"}
Jul 17 10:05:30.461714 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:30.461405    2574 scope.go:117] "RemoveContainer" containerID="f955cb43fb46797ceaed6e28fc161a3ce4d0521a391e4ab31abe6e0e729bd824"
Jul 17 10:05:30.461714 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:30.461554    2574 scope.go:117] "RemoveContainer" containerID="32b182873c2f8449e9e84ac35de46f52af6baa939ea723320e40cc4d9597eeb3"
Jul 17 10:05:30.461899 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:05:30.461805    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-9tbsj_odh-ai-gateway-infra(575bac5c-c3c4-4742-8b20-b5b6e575fb06)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9tbsj" podUID="575bac5c-c3c4-4742-8b20-b5b6e575fb06"
Jul 17 10:05:31.468856 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:31.468829    2574 scope.go:117] "RemoveContainer" containerID="32b182873c2f8449e9e84ac35de46f52af6baa939ea723320e40cc4d9597eeb3"
Jul 17 10:05:31.469278 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:05:31.469043    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-9tbsj_odh-ai-gateway-infra(575bac5c-c3c4-4742-8b20-b5b6e575fb06)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9tbsj" podUID="575bac5c-c3c4-4742-8b20-b5b6e575fb06"
Jul 17 10:05:35.940260 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:35.940157    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bvpqc"]
Jul 17 10:05:35.940657 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:35.940612    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f4a870f-e7d9-4729-bfe9-c147c732cec1" containerName="revoke-keys"
Jul 17 10:05:35.940657 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:35.940625    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f4a870f-e7d9-4729-bfe9-c147c732cec1" containerName="revoke-keys"
Jul 17 10:05:35.940657 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:35.940638    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f019bdd-ac0e-4dac-a3c7-dc24f72d083d" containerName="revoke-keys"
Jul 17 10:05:35.940657 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:35.940644    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f019bdd-ac0e-4dac-a3c7-dc24f72d083d" containerName="revoke-keys"
Jul 17 10:05:35.940813 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:35.940702    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f019bdd-ac0e-4dac-a3c7-dc24f72d083d" containerName="revoke-keys"
Jul 17 10:05:35.940813 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:35.940710    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f4a870f-e7d9-4729-bfe9-c147c732cec1" containerName="revoke-keys"
Jul 17 10:05:35.945112 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:35.945095    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bvpqc"
Jul 17 10:05:35.951136 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:35.950698    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bvpqc"]
Jul 17 10:05:36.083899 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:36.083859    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11897688-a1f4-4846-9fd9-70743d1389a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-bvpqc\" (UID: \"11897688-a1f4-4846-9fd9-70743d1389a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bvpqc"
Jul 17 10:05:36.184528 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:36.184495    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11897688-a1f4-4846-9fd9-70743d1389a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-bvpqc\" (UID: \"11897688-a1f4-4846-9fd9-70743d1389a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bvpqc"
Jul 17 10:05:36.185166 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:36.185144    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11897688-a1f4-4846-9fd9-70743d1389a0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-bvpqc\" (UID: \"11897688-a1f4-4846-9fd9-70743d1389a0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bvpqc"
Jul 17 10:05:36.258962 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:36.258869    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bvpqc"
Jul 17 10:05:36.400992 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:36.400970    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bvpqc"]
Jul 17 10:05:36.403380 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:05:36.403353    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11897688_a1f4_4846_9fd9_70743d1389a0.slice/crio-37ab96f7307c888e1f5b06f44206ef2745856752dc946aafdb59d7073e633f51 WatchSource:0}: Error finding container 37ab96f7307c888e1f5b06f44206ef2745856752dc946aafdb59d7073e633f51: Status 404 returned error can't find the container with id 37ab96f7307c888e1f5b06f44206ef2745856752dc946aafdb59d7073e633f51
Jul 17 10:05:36.502690 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:36.502651    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bvpqc" event={"ID":"11897688-a1f4-4846-9fd9-70743d1389a0","Type":"ContainerStarted","Data":"37ab96f7307c888e1f5b06f44206ef2745856752dc946aafdb59d7073e633f51"}
Jul 17 10:05:37.509924 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:37.509888    2574 generic.go:356] "Generic (PLEG): container finished" podID="11897688-a1f4-4846-9fd9-70743d1389a0" containerID="789ef9dccae3c4d711027adefc9c2f57de7ff2c85c96fbd406c014994244f756" exitCode=6
Jul 17 10:05:37.510363 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:37.509981    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bvpqc" event={"ID":"11897688-a1f4-4846-9fd9-70743d1389a0","Type":"ContainerDied","Data":"789ef9dccae3c4d711027adefc9c2f57de7ff2c85c96fbd406c014994244f756"}
Jul 17 10:05:37.510363 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:37.510155    2574 scope.go:117] "RemoveContainer" containerID="789ef9dccae3c4d711027adefc9c2f57de7ff2c85c96fbd406c014994244f756"
Jul 17 10:05:38.517487 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:38.517445    2574 generic.go:356] "Generic (PLEG): container finished" podID="11897688-a1f4-4846-9fd9-70743d1389a0" containerID="0c1784828a71d4329113d9ccf7a260ee651d88dcdcb101b2a19b11b4cfb3537d" exitCode=6
Jul 17 10:05:38.517487 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:38.517482    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bvpqc" event={"ID":"11897688-a1f4-4846-9fd9-70743d1389a0","Type":"ContainerDied","Data":"0c1784828a71d4329113d9ccf7a260ee651d88dcdcb101b2a19b11b4cfb3537d"}
Jul 17 10:05:38.518029 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:38.517529    2574 scope.go:117] "RemoveContainer" containerID="789ef9dccae3c4d711027adefc9c2f57de7ff2c85c96fbd406c014994244f756"
Jul 17 10:05:38.518029 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:38.517781    2574 scope.go:117] "RemoveContainer" containerID="0c1784828a71d4329113d9ccf7a260ee651d88dcdcb101b2a19b11b4cfb3537d"
Jul 17 10:05:38.518136 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:05:38.518050    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-bvpqc_odh-ai-gateway-infra(11897688-a1f4-4846-9fd9-70743d1389a0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bvpqc" podUID="11897688-a1f4-4846-9fd9-70743d1389a0"
Jul 17 10:05:39.524054 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:39.524026    2574 scope.go:117] "RemoveContainer" containerID="0c1784828a71d4329113d9ccf7a260ee651d88dcdcb101b2a19b11b4cfb3537d"
Jul 17 10:05:39.524548 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:05:39.524318    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-bvpqc_odh-ai-gateway-infra(11897688-a1f4-4846-9fd9-70743d1389a0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bvpqc" podUID="11897688-a1f4-4846-9fd9-70743d1389a0"
Jul 17 10:05:42.893807 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:42.893768    2574 scope.go:117] "RemoveContainer" containerID="32b182873c2f8449e9e84ac35de46f52af6baa939ea723320e40cc4d9597eeb3"
Jul 17 10:05:43.547042 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:43.547013    2574 generic.go:356] "Generic (PLEG): container finished" podID="575bac5c-c3c4-4742-8b20-b5b6e575fb06" containerID="ffd12c42b95e350108e37c39a39ea87c7a68ae716c6ce1543f4e4ac7f2f8940c" exitCode=6
Jul 17 10:05:43.547175 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:43.547090    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9tbsj" event={"ID":"575bac5c-c3c4-4742-8b20-b5b6e575fb06","Type":"ContainerDied","Data":"ffd12c42b95e350108e37c39a39ea87c7a68ae716c6ce1543f4e4ac7f2f8940c"}
Jul 17 10:05:43.547175 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:43.547171    2574 scope.go:117] "RemoveContainer" containerID="32b182873c2f8449e9e84ac35de46f52af6baa939ea723320e40cc4d9597eeb3"
Jul 17 10:05:43.547478 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:43.547457    2574 scope.go:117] "RemoveContainer" containerID="ffd12c42b95e350108e37c39a39ea87c7a68ae716c6ce1543f4e4ac7f2f8940c"
Jul 17 10:05:43.547790 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:05:43.547738    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-9tbsj_odh-ai-gateway-infra(575bac5c-c3c4-4742-8b20-b5b6e575fb06)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9tbsj" podUID="575bac5c-c3c4-4742-8b20-b5b6e575fb06"
Jul 17 10:05:44.574119 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:44.574081    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9tbsj"]
Jul 17 10:05:44.740491 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:44.740466    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9tbsj"
Jul 17 10:05:44.874476 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:44.874389    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/575bac5c-c3c4-4742-8b20-b5b6e575fb06-maas-api-service-ca\") pod \"575bac5c-c3c4-4742-8b20-b5b6e575fb06\" (UID: \"575bac5c-c3c4-4742-8b20-b5b6e575fb06\") "
Jul 17 10:05:44.874751 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:44.874728    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/575bac5c-c3c4-4742-8b20-b5b6e575fb06-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "575bac5c-c3c4-4742-8b20-b5b6e575fb06" (UID: "575bac5c-c3c4-4742-8b20-b5b6e575fb06"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:05:44.975989 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:44.975956    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/575bac5c-c3c4-4742-8b20-b5b6e575fb06-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:05:45.561268 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:45.561203    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9tbsj" event={"ID":"575bac5c-c3c4-4742-8b20-b5b6e575fb06","Type":"ContainerDied","Data":"9622e54e38807d978a37841fd38f97246ffaf01ca1b154a396a1219a933afa17"}
Jul 17 10:05:45.561268 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:45.561271    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9tbsj"
Jul 17 10:05:45.561550 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:45.561295    2574 scope.go:117] "RemoveContainer" containerID="ffd12c42b95e350108e37c39a39ea87c7a68ae716c6ce1543f4e4ac7f2f8940c"
Jul 17 10:05:45.577613 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:45.577589    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9tbsj"]
Jul 17 10:05:45.582124 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:45.582102    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-9tbsj"]
Jul 17 10:05:46.898020 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:46.897990    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="575bac5c-c3c4-4742-8b20-b5b6e575fb06" path="/var/lib/kubelet/pods/575bac5c-c3c4-4742-8b20-b5b6e575fb06/volumes"
Jul 17 10:05:46.902514 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:46.902491    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-s8t8r"]
Jul 17 10:05:46.902903 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:46.902891    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="575bac5c-c3c4-4742-8b20-b5b6e575fb06" containerName="revoke-keys"
Jul 17 10:05:46.902951 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:46.902905    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="575bac5c-c3c4-4742-8b20-b5b6e575fb06" containerName="revoke-keys"
Jul 17 10:05:46.902951 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:46.902913    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="575bac5c-c3c4-4742-8b20-b5b6e575fb06" containerName="revoke-keys"
Jul 17 10:05:46.902951 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:46.902918    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="575bac5c-c3c4-4742-8b20-b5b6e575fb06" containerName="revoke-keys"
Jul 17 10:05:46.903050 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:46.902997    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="575bac5c-c3c4-4742-8b20-b5b6e575fb06" containerName="revoke-keys"
Jul 17 10:05:46.903050 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:46.903005    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="575bac5c-c3c4-4742-8b20-b5b6e575fb06" containerName="revoke-keys"
Jul 17 10:05:46.903050 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:46.903012    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="575bac5c-c3c4-4742-8b20-b5b6e575fb06" containerName="revoke-keys"
Jul 17 10:05:46.907637 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:46.907619    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-s8t8r"
Jul 17 10:05:46.916592 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:46.916564    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-s8t8r"]
Jul 17 10:05:46.996315 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:46.996227    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed6d9b40-0168-4c2b-ad38-199c7a657670-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-s8t8r\" (UID: \"ed6d9b40-0168-4c2b-ad38-199c7a657670\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-s8t8r"
Jul 17 10:05:47.097143 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:47.097101    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed6d9b40-0168-4c2b-ad38-199c7a657670-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-s8t8r\" (UID: \"ed6d9b40-0168-4c2b-ad38-199c7a657670\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-s8t8r"
Jul 17 10:05:47.097873 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:47.097845    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed6d9b40-0168-4c2b-ad38-199c7a657670-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-s8t8r\" (UID: \"ed6d9b40-0168-4c2b-ad38-199c7a657670\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-s8t8r"
Jul 17 10:05:47.222340 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:47.222307    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-s8t8r"
Jul 17 10:05:47.362298 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:47.362172    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-s8t8r"]
Jul 17 10:05:47.365100 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:05:47.365058    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded6d9b40_0168_4c2b_ad38_199c7a657670.slice/crio-43396bc28b289b7360aa388cc5fa13f3e125de151d6fba05e6935c41798679c9 WatchSource:0}: Error finding container 43396bc28b289b7360aa388cc5fa13f3e125de151d6fba05e6935c41798679c9: Status 404 returned error can't find the container with id 43396bc28b289b7360aa388cc5fa13f3e125de151d6fba05e6935c41798679c9
Jul 17 10:05:47.575050 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:47.574958    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-s8t8r" event={"ID":"ed6d9b40-0168-4c2b-ad38-199c7a657670","Type":"ContainerStarted","Data":"43396bc28b289b7360aa388cc5fa13f3e125de151d6fba05e6935c41798679c9"}
Jul 17 10:05:48.582259 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:48.582205    2574 generic.go:356] "Generic (PLEG): container finished" podID="ed6d9b40-0168-4c2b-ad38-199c7a657670" containerID="2fbbc3fe3edaa44e79d5fde47eb3263aabfe2aabccca910de3472850ac3b4076" exitCode=6
Jul 17 10:05:48.582709 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:48.582296    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-s8t8r" event={"ID":"ed6d9b40-0168-4c2b-ad38-199c7a657670","Type":"ContainerDied","Data":"2fbbc3fe3edaa44e79d5fde47eb3263aabfe2aabccca910de3472850ac3b4076"}
Jul 17 10:05:48.582709 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:48.582516    2574 scope.go:117] "RemoveContainer" containerID="2fbbc3fe3edaa44e79d5fde47eb3263aabfe2aabccca910de3472850ac3b4076"
Jul 17 10:05:49.590124 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:49.590034    2574 generic.go:356] "Generic (PLEG): container finished" podID="ed6d9b40-0168-4c2b-ad38-199c7a657670" containerID="4bcfe83d1ea84a4795478974c4f3e97e222f4d9a8596a727597f17991d2f829c" exitCode=6
Jul 17 10:05:49.590124 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:49.590091    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-s8t8r" event={"ID":"ed6d9b40-0168-4c2b-ad38-199c7a657670","Type":"ContainerDied","Data":"4bcfe83d1ea84a4795478974c4f3e97e222f4d9a8596a727597f17991d2f829c"}
Jul 17 10:05:49.590600 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:49.590155    2574 scope.go:117] "RemoveContainer" containerID="2fbbc3fe3edaa44e79d5fde47eb3263aabfe2aabccca910de3472850ac3b4076"
Jul 17 10:05:49.590600 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:49.590336    2574 scope.go:117] "RemoveContainer" containerID="4bcfe83d1ea84a4795478974c4f3e97e222f4d9a8596a727597f17991d2f829c"
Jul 17 10:05:49.590678 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:05:49.590606    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-s8t8r_odh-ai-gateway-infra(ed6d9b40-0168-4c2b-ad38-199c7a657670)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-s8t8r" podUID="ed6d9b40-0168-4c2b-ad38-199c7a657670"
Jul 17 10:05:50.597945 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:50.597916    2574 scope.go:117] "RemoveContainer" containerID="4bcfe83d1ea84a4795478974c4f3e97e222f4d9a8596a727597f17991d2f829c"
Jul 17 10:05:50.598383 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:05:50.598115    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-s8t8r_odh-ai-gateway-infra(ed6d9b40-0168-4c2b-ad38-199c7a657670)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-s8t8r" podUID="ed6d9b40-0168-4c2b-ad38-199c7a657670"
Jul 17 10:05:53.891442 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:53.891409    2574 scope.go:117] "RemoveContainer" containerID="0c1784828a71d4329113d9ccf7a260ee651d88dcdcb101b2a19b11b4cfb3537d"
Jul 17 10:05:54.620757 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:54.620732    2574 generic.go:356] "Generic (PLEG): container finished" podID="11897688-a1f4-4846-9fd9-70743d1389a0" containerID="b9f6c1685efe4d6afcf9a07afc9fe4731c4f6134efaf831c30397eb156f607e3" exitCode=6
Jul 17 10:05:54.620861 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:54.620799    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bvpqc" event={"ID":"11897688-a1f4-4846-9fd9-70743d1389a0","Type":"ContainerDied","Data":"b9f6c1685efe4d6afcf9a07afc9fe4731c4f6134efaf831c30397eb156f607e3"}
Jul 17 10:05:54.620908 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:54.620860    2574 scope.go:117] "RemoveContainer" containerID="0c1784828a71d4329113d9ccf7a260ee651d88dcdcb101b2a19b11b4cfb3537d"
Jul 17 10:05:54.621222 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:54.621201    2574 scope.go:117] "RemoveContainer" containerID="b9f6c1685efe4d6afcf9a07afc9fe4731c4f6134efaf831c30397eb156f607e3"
Jul 17 10:05:54.621459 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:05:54.621435    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-bvpqc_odh-ai-gateway-infra(11897688-a1f4-4846-9fd9-70743d1389a0)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bvpqc" podUID="11897688-a1f4-4846-9fd9-70743d1389a0"
Jul 17 10:05:55.647603 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:55.647569    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bvpqc"]
Jul 17 10:05:55.797214 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:55.797191    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bvpqc"
Jul 17 10:05:55.884689 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:55.884655    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11897688-a1f4-4846-9fd9-70743d1389a0-maas-api-service-ca\") pod \"11897688-a1f4-4846-9fd9-70743d1389a0\" (UID: \"11897688-a1f4-4846-9fd9-70743d1389a0\") "
Jul 17 10:05:55.884996 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:55.884974    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11897688-a1f4-4846-9fd9-70743d1389a0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "11897688-a1f4-4846-9fd9-70743d1389a0" (UID: "11897688-a1f4-4846-9fd9-70743d1389a0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:05:55.986182 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:55.986149    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11897688-a1f4-4846-9fd9-70743d1389a0-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:05:56.637763 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:56.637736    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bvpqc"
Jul 17 10:05:56.637763 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:56.637748    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bvpqc" event={"ID":"11897688-a1f4-4846-9fd9-70743d1389a0","Type":"ContainerDied","Data":"37ab96f7307c888e1f5b06f44206ef2745856752dc946aafdb59d7073e633f51"}
Jul 17 10:05:56.638059 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:56.637819    2574 scope.go:117] "RemoveContainer" containerID="b9f6c1685efe4d6afcf9a07afc9fe4731c4f6134efaf831c30397eb156f607e3"
Jul 17 10:05:56.670371 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:56.667447    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bvpqc"]
Jul 17 10:05:56.676025 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:56.675995    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-bvpqc"]
Jul 17 10:05:56.898405 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:05:56.898323    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11897688-a1f4-4846-9fd9-70743d1389a0" path="/var/lib/kubelet/pods/11897688-a1f4-4846-9fd9-70743d1389a0/volumes"
Jul 17 10:06:01.891312 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:01.891271    2574 scope.go:117] "RemoveContainer" containerID="4bcfe83d1ea84a4795478974c4f3e97e222f4d9a8596a727597f17991d2f829c"
Jul 17 10:06:02.672997 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:02.672968    2574 generic.go:356] "Generic (PLEG): container finished" podID="ed6d9b40-0168-4c2b-ad38-199c7a657670" containerID="f7de5e751d72add0e35348441baf4056bc3b659c4fc0ada7047e2dfc63793b8a" exitCode=6
Jul 17 10:06:02.673152 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:02.673039    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-s8t8r" event={"ID":"ed6d9b40-0168-4c2b-ad38-199c7a657670","Type":"ContainerDied","Data":"f7de5e751d72add0e35348441baf4056bc3b659c4fc0ada7047e2dfc63793b8a"}
Jul 17 10:06:02.673152 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:02.673102    2574 scope.go:117] "RemoveContainer" containerID="4bcfe83d1ea84a4795478974c4f3e97e222f4d9a8596a727597f17991d2f829c"
Jul 17 10:06:02.673383 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:02.673370    2574 scope.go:117] "RemoveContainer" containerID="f7de5e751d72add0e35348441baf4056bc3b659c4fc0ada7047e2dfc63793b8a"
Jul 17 10:06:02.673656 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:06:02.673630    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-s8t8r_odh-ai-gateway-infra(ed6d9b40-0168-4c2b-ad38-199c7a657670)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-s8t8r" podUID="ed6d9b40-0168-4c2b-ad38-199c7a657670"
Jul 17 10:06:03.701198 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:03.701158    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-s8t8r"]
Jul 17 10:06:03.849436 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:03.849404    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-s8t8r"
Jul 17 10:06:03.954013 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:03.953935    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed6d9b40-0168-4c2b-ad38-199c7a657670-maas-api-service-ca\") pod \"ed6d9b40-0168-4c2b-ad38-199c7a657670\" (UID: \"ed6d9b40-0168-4c2b-ad38-199c7a657670\") "
Jul 17 10:06:03.954295 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:03.954270    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed6d9b40-0168-4c2b-ad38-199c7a657670-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ed6d9b40-0168-4c2b-ad38-199c7a657670" (UID: "ed6d9b40-0168-4c2b-ad38-199c7a657670"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:06:04.055544 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:04.055510    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed6d9b40-0168-4c2b-ad38-199c7a657670-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:06:04.686682 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:04.686658    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-s8t8r"
Jul 17 10:06:04.686682 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:04.686667    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-s8t8r" event={"ID":"ed6d9b40-0168-4c2b-ad38-199c7a657670","Type":"ContainerDied","Data":"43396bc28b289b7360aa388cc5fa13f3e125de151d6fba05e6935c41798679c9"}
Jul 17 10:06:04.686890 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:04.686724    2574 scope.go:117] "RemoveContainer" containerID="f7de5e751d72add0e35348441baf4056bc3b659c4fc0ada7047e2dfc63793b8a"
Jul 17 10:06:04.710080 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:04.710056    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-s8t8r"]
Jul 17 10:06:04.716869 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:04.716846    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-s8t8r"]
Jul 17 10:06:04.897859 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:04.897822    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed6d9b40-0168-4c2b-ad38-199c7a657670" path="/var/lib/kubelet/pods/ed6d9b40-0168-4c2b-ad38-199c7a657670/volumes"
Jul 17 10:06:05.366926 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:05.366881    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-2rrgl"]
Jul 17 10:06:05.367359 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:05.367332    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11897688-a1f4-4846-9fd9-70743d1389a0" containerName="revoke-keys"
Jul 17 10:06:05.367359 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:05.367349    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="11897688-a1f4-4846-9fd9-70743d1389a0" containerName="revoke-keys"
Jul 17 10:06:05.367535 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:05.367364    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11897688-a1f4-4846-9fd9-70743d1389a0" containerName="revoke-keys"
Jul 17 10:06:05.367535 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:05.367373    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="11897688-a1f4-4846-9fd9-70743d1389a0" containerName="revoke-keys"
Jul 17 10:06:05.367535 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:05.367384    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed6d9b40-0168-4c2b-ad38-199c7a657670" containerName="revoke-keys"
Jul 17 10:06:05.367535 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:05.367390    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed6d9b40-0168-4c2b-ad38-199c7a657670" containerName="revoke-keys"
Jul 17 10:06:05.367535 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:05.367406    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="575bac5c-c3c4-4742-8b20-b5b6e575fb06" containerName="revoke-keys"
Jul 17 10:06:05.367535 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:05.367412    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="575bac5c-c3c4-4742-8b20-b5b6e575fb06" containerName="revoke-keys"
Jul 17 10:06:05.367535 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:05.367418    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed6d9b40-0168-4c2b-ad38-199c7a657670" containerName="revoke-keys"
Jul 17 10:06:05.367535 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:05.367423    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed6d9b40-0168-4c2b-ad38-199c7a657670" containerName="revoke-keys"
Jul 17 10:06:05.367535 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:05.367496    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="11897688-a1f4-4846-9fd9-70743d1389a0" containerName="revoke-keys"
Jul 17 10:06:05.367535 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:05.367505    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed6d9b40-0168-4c2b-ad38-199c7a657670" containerName="revoke-keys"
Jul 17 10:06:05.367535 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:05.367511    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="11897688-a1f4-4846-9fd9-70743d1389a0" containerName="revoke-keys"
Jul 17 10:06:05.367535 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:05.367517    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed6d9b40-0168-4c2b-ad38-199c7a657670" containerName="revoke-keys"
Jul 17 10:06:05.367535 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:05.367526    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed6d9b40-0168-4c2b-ad38-199c7a657670" containerName="revoke-keys"
Jul 17 10:06:05.372222 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:05.372200    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-2rrgl"
Jul 17 10:06:05.375406 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:05.375206    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:06:05.378346 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:05.378320    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-2rrgl"]
Jul 17 10:06:05.470162 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:05.470125    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b24c628-2cd0-4b36-a272-ebc5d3941287-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-2rrgl\" (UID: \"4b24c628-2cd0-4b36-a272-ebc5d3941287\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-2rrgl"
Jul 17 10:06:05.571429 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:05.571395    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b24c628-2cd0-4b36-a272-ebc5d3941287-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-2rrgl\" (UID: \"4b24c628-2cd0-4b36-a272-ebc5d3941287\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-2rrgl"
Jul 17 10:06:05.572033 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:05.572014    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b24c628-2cd0-4b36-a272-ebc5d3941287-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-2rrgl\" (UID: \"4b24c628-2cd0-4b36-a272-ebc5d3941287\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-2rrgl"
Jul 17 10:06:05.686407 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:05.686381    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-2rrgl"
Jul 17 10:06:05.827935 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:05.827911    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-2rrgl"]
Jul 17 10:06:05.830498 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:06:05.830468    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b24c628_2cd0_4b36_a272_ebc5d3941287.slice/crio-ef25a8ce63a0d3bb453b6be24ce540ab503632a6a91158a6bf32103866ae5cd0 WatchSource:0}: Error finding container ef25a8ce63a0d3bb453b6be24ce540ab503632a6a91158a6bf32103866ae5cd0: Status 404 returned error can't find the container with id ef25a8ce63a0d3bb453b6be24ce540ab503632a6a91158a6bf32103866ae5cd0
Jul 17 10:06:06.700774 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:06.700735    2574 generic.go:356] "Generic (PLEG): container finished" podID="4b24c628-2cd0-4b36-a272-ebc5d3941287" containerID="65cb09cf3f7a17049308b928cf36fa39652e3cd86dff146553d1f995502cc588" exitCode=7
Jul 17 10:06:06.700973 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:06.700784    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-2rrgl" event={"ID":"4b24c628-2cd0-4b36-a272-ebc5d3941287","Type":"ContainerDied","Data":"65cb09cf3f7a17049308b928cf36fa39652e3cd86dff146553d1f995502cc588"}
Jul 17 10:06:06.700973 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:06.700831    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-2rrgl" event={"ID":"4b24c628-2cd0-4b36-a272-ebc5d3941287","Type":"ContainerStarted","Data":"ef25a8ce63a0d3bb453b6be24ce540ab503632a6a91158a6bf32103866ae5cd0"}
Jul 17 10:06:06.701061 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:06.701001    2574 scope.go:117] "RemoveContainer" containerID="65cb09cf3f7a17049308b928cf36fa39652e3cd86dff146553d1f995502cc588"
Jul 17 10:06:07.707981 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:07.707939    2574 generic.go:356] "Generic (PLEG): container finished" podID="4b24c628-2cd0-4b36-a272-ebc5d3941287" containerID="b3fd566427404f879a9b5282f6dc6852e9d9437c3d295b9a063f54e4934662c9" exitCode=7
Jul 17 10:06:07.708530 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:07.708002    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-2rrgl" event={"ID":"4b24c628-2cd0-4b36-a272-ebc5d3941287","Type":"ContainerDied","Data":"b3fd566427404f879a9b5282f6dc6852e9d9437c3d295b9a063f54e4934662c9"}
Jul 17 10:06:07.708530 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:07.708088    2574 scope.go:117] "RemoveContainer" containerID="65cb09cf3f7a17049308b928cf36fa39652e3cd86dff146553d1f995502cc588"
Jul 17 10:06:07.708530 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:07.708396    2574 scope.go:117] "RemoveContainer" containerID="b3fd566427404f879a9b5282f6dc6852e9d9437c3d295b9a063f54e4934662c9"
Jul 17 10:06:07.708699 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:06:07.708634    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-2rrgl_odh-ai-gateway-infra(4b24c628-2cd0-4b36-a272-ebc5d3941287)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-2rrgl" podUID="4b24c628-2cd0-4b36-a272-ebc5d3941287"
Jul 17 10:06:08.715832 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:08.715806    2574 scope.go:117] "RemoveContainer" containerID="b3fd566427404f879a9b5282f6dc6852e9d9437c3d295b9a063f54e4934662c9"
Jul 17 10:06:08.716362 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:06:08.716085    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-2rrgl_odh-ai-gateway-infra(4b24c628-2cd0-4b36-a272-ebc5d3941287)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-2rrgl" podUID="4b24c628-2cd0-4b36-a272-ebc5d3941287"
Jul 17 10:06:18.084968 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:18.084935    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpzd6"]
Jul 17 10:06:18.085476 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:18.085383    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed6d9b40-0168-4c2b-ad38-199c7a657670" containerName="revoke-keys"
Jul 17 10:06:18.085476 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:18.085395    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed6d9b40-0168-4c2b-ad38-199c7a657670" containerName="revoke-keys"
Jul 17 10:06:18.085476 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:18.085413    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11897688-a1f4-4846-9fd9-70743d1389a0" containerName="revoke-keys"
Jul 17 10:06:18.085476 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:18.085419    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="11897688-a1f4-4846-9fd9-70743d1389a0" containerName="revoke-keys"
Jul 17 10:06:18.085610 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:18.085489    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="11897688-a1f4-4846-9fd9-70743d1389a0" containerName="revoke-keys"
Jul 17 10:06:18.088653 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:18.088636    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpzd6"
Jul 17 10:06:18.098663 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:18.098636    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpzd6"]
Jul 17 10:06:18.192763 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:18.192720    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2283bf9-6253-425f-b801-10fce8c6274e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-rpzd6\" (UID: \"f2283bf9-6253-425f-b801-10fce8c6274e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpzd6"
Jul 17 10:06:18.293710 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:18.293671    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2283bf9-6253-425f-b801-10fce8c6274e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-rpzd6\" (UID: \"f2283bf9-6253-425f-b801-10fce8c6274e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpzd6"
Jul 17 10:06:18.294311 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:18.294292    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2283bf9-6253-425f-b801-10fce8c6274e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-rpzd6\" (UID: \"f2283bf9-6253-425f-b801-10fce8c6274e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpzd6"
Jul 17 10:06:18.399903 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:18.399817    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpzd6"
Jul 17 10:06:18.751287 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:18.751256    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpzd6"]
Jul 17 10:06:18.753321 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:06:18.753294    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2283bf9_6253_425f_b801_10fce8c6274e.slice/crio-a31a13710a5b872579c942d240e7d08099809666582e485d3d947ed3bca396e8 WatchSource:0}: Error finding container a31a13710a5b872579c942d240e7d08099809666582e485d3d947ed3bca396e8: Status 404 returned error can't find the container with id a31a13710a5b872579c942d240e7d08099809666582e485d3d947ed3bca396e8
Jul 17 10:06:18.771029 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:18.771004    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpzd6" event={"ID":"f2283bf9-6253-425f-b801-10fce8c6274e","Type":"ContainerStarted","Data":"a31a13710a5b872579c942d240e7d08099809666582e485d3d947ed3bca396e8"}
Jul 17 10:06:19.784708 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:19.784669    2574 generic.go:356] "Generic (PLEG): container finished" podID="f2283bf9-6253-425f-b801-10fce8c6274e" containerID="b5968b0ffb890847342a5ae684dc26f5d32ce0925eba29f4cb42554e495b81e6" exitCode=6
Jul 17 10:06:19.785198 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:19.784753    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpzd6" event={"ID":"f2283bf9-6253-425f-b801-10fce8c6274e","Type":"ContainerDied","Data":"b5968b0ffb890847342a5ae684dc26f5d32ce0925eba29f4cb42554e495b81e6"}
Jul 17 10:06:19.785198 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:19.784975    2574 scope.go:117] "RemoveContainer" containerID="b5968b0ffb890847342a5ae684dc26f5d32ce0925eba29f4cb42554e495b81e6"
Jul 17 10:06:20.792503 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:20.792463    2574 generic.go:356] "Generic (PLEG): container finished" podID="f2283bf9-6253-425f-b801-10fce8c6274e" containerID="9ca6eaa57938d19219c8a0d6c8d2bb25f9af9c76ce55d3e99ccf454bd05316fa" exitCode=6
Jul 17 10:06:20.792952 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:20.792518    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpzd6" event={"ID":"f2283bf9-6253-425f-b801-10fce8c6274e","Type":"ContainerDied","Data":"9ca6eaa57938d19219c8a0d6c8d2bb25f9af9c76ce55d3e99ccf454bd05316fa"}
Jul 17 10:06:20.792952 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:20.792578    2574 scope.go:117] "RemoveContainer" containerID="b5968b0ffb890847342a5ae684dc26f5d32ce0925eba29f4cb42554e495b81e6"
Jul 17 10:06:20.792952 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:20.792753    2574 scope.go:117] "RemoveContainer" containerID="9ca6eaa57938d19219c8a0d6c8d2bb25f9af9c76ce55d3e99ccf454bd05316fa"
Jul 17 10:06:20.793113 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:06:20.793008    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-rpzd6_odh-ai-gateway-infra(f2283bf9-6253-425f-b801-10fce8c6274e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpzd6" podUID="f2283bf9-6253-425f-b801-10fce8c6274e"
Jul 17 10:06:20.891909 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:20.891867    2574 scope.go:117] "RemoveContainer" containerID="b3fd566427404f879a9b5282f6dc6852e9d9437c3d295b9a063f54e4934662c9"
Jul 17 10:06:21.799531 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:21.799451    2574 generic.go:356] "Generic (PLEG): container finished" podID="4b24c628-2cd0-4b36-a272-ebc5d3941287" containerID="2814efe9fcdd245bf53b6d325887ae0362c8bfb05bc77e83b29dcad326b7b397" exitCode=7
Jul 17 10:06:21.799531 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:21.799520    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-2rrgl" event={"ID":"4b24c628-2cd0-4b36-a272-ebc5d3941287","Type":"ContainerDied","Data":"2814efe9fcdd245bf53b6d325887ae0362c8bfb05bc77e83b29dcad326b7b397"}
Jul 17 10:06:21.799973 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:21.799581    2574 scope.go:117] "RemoveContainer" containerID="b3fd566427404f879a9b5282f6dc6852e9d9437c3d295b9a063f54e4934662c9"
Jul 17 10:06:21.799973 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:21.799939    2574 scope.go:117] "RemoveContainer" containerID="2814efe9fcdd245bf53b6d325887ae0362c8bfb05bc77e83b29dcad326b7b397"
Jul 17 10:06:21.800203 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:06:21.800178    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-2rrgl_odh-ai-gateway-infra(4b24c628-2cd0-4b36-a272-ebc5d3941287)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-2rrgl" podUID="4b24c628-2cd0-4b36-a272-ebc5d3941287"
Jul 17 10:06:21.801447 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:21.801428    2574 scope.go:117] "RemoveContainer" containerID="9ca6eaa57938d19219c8a0d6c8d2bb25f9af9c76ce55d3e99ccf454bd05316fa"
Jul 17 10:06:21.801614 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:06:21.801597    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-rpzd6_odh-ai-gateway-infra(f2283bf9-6253-425f-b801-10fce8c6274e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpzd6" podUID="f2283bf9-6253-425f-b801-10fce8c6274e"
Jul 17 10:06:22.830871 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:22.830836    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-2rrgl"]
Jul 17 10:06:22.974353 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:22.974329    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-2rrgl"
Jul 17 10:06:23.040471 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:23.040440    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b24c628-2cd0-4b36-a272-ebc5d3941287-maas-api-service-ca\") pod \"4b24c628-2cd0-4b36-a272-ebc5d3941287\" (UID: \"4b24c628-2cd0-4b36-a272-ebc5d3941287\") "
Jul 17 10:06:23.040773 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:23.040751    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b24c628-2cd0-4b36-a272-ebc5d3941287-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4b24c628-2cd0-4b36-a272-ebc5d3941287" (UID: "4b24c628-2cd0-4b36-a272-ebc5d3941287"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:06:23.141766 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:23.141672    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b24c628-2cd0-4b36-a272-ebc5d3941287-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:06:23.815426 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:23.815396    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-2rrgl"
Jul 17 10:06:23.815621 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:23.815424    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-2rrgl" event={"ID":"4b24c628-2cd0-4b36-a272-ebc5d3941287","Type":"ContainerDied","Data":"ef25a8ce63a0d3bb453b6be24ce540ab503632a6a91158a6bf32103866ae5cd0"}
Jul 17 10:06:23.815621 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:23.815490    2574 scope.go:117] "RemoveContainer" containerID="2814efe9fcdd245bf53b6d325887ae0362c8bfb05bc77e83b29dcad326b7b397"
Jul 17 10:06:23.838720 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:23.838695    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-2rrgl"]
Jul 17 10:06:23.844540 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:23.844517    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-2rrgl"]
Jul 17 10:06:24.898114 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:24.898083    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b24c628-2cd0-4b36-a272-ebc5d3941287" path="/var/lib/kubelet/pods/4b24c628-2cd0-4b36-a272-ebc5d3941287/volumes"
Jul 17 10:06:26.087327 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:26.087291    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6c5c6b98f-rlrg6_8ab844e0-7c94-4003-9e42-ada02b05d4e9/manager/0.log"
Jul 17 10:06:32.894410 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:32.894375    2574 scope.go:117] "RemoveContainer" containerID="9ca6eaa57938d19219c8a0d6c8d2bb25f9af9c76ce55d3e99ccf454bd05316fa"
Jul 17 10:06:33.872122 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:33.872030    2574 generic.go:356] "Generic (PLEG): container finished" podID="f2283bf9-6253-425f-b801-10fce8c6274e" containerID="d3e2705dc1c2065928c845d3e2972ef95945c1ce3b25b642c896eb26156e66f6" exitCode=6
Jul 17 10:06:33.872122 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:33.872076    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpzd6" event={"ID":"f2283bf9-6253-425f-b801-10fce8c6274e","Type":"ContainerDied","Data":"d3e2705dc1c2065928c845d3e2972ef95945c1ce3b25b642c896eb26156e66f6"}
Jul 17 10:06:33.872384 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:33.872154    2574 scope.go:117] "RemoveContainer" containerID="9ca6eaa57938d19219c8a0d6c8d2bb25f9af9c76ce55d3e99ccf454bd05316fa"
Jul 17 10:06:33.872492 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:33.872474    2574 scope.go:117] "RemoveContainer" containerID="d3e2705dc1c2065928c845d3e2972ef95945c1ce3b25b642c896eb26156e66f6"
Jul 17 10:06:33.872771 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:06:33.872747    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-rpzd6_odh-ai-gateway-infra(f2283bf9-6253-425f-b801-10fce8c6274e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpzd6" podUID="f2283bf9-6253-425f-b801-10fce8c6274e"
Jul 17 10:06:34.899553 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:34.899519    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpzd6"]
Jul 17 10:06:35.045328 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:35.045303    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpzd6"
Jul 17 10:06:35.164529 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:35.164504    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2283bf9-6253-425f-b801-10fce8c6274e-maas-api-service-ca\") pod \"f2283bf9-6253-425f-b801-10fce8c6274e\" (UID: \"f2283bf9-6253-425f-b801-10fce8c6274e\") "
Jul 17 10:06:35.164869 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:35.164851    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2283bf9-6253-425f-b801-10fce8c6274e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f2283bf9-6253-425f-b801-10fce8c6274e" (UID: "f2283bf9-6253-425f-b801-10fce8c6274e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:06:35.265701 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:35.265668    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2283bf9-6253-425f-b801-10fce8c6274e-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:06:35.886478 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:35.886446    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpzd6"
Jul 17 10:06:35.886669 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:35.886450    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpzd6" event={"ID":"f2283bf9-6253-425f-b801-10fce8c6274e","Type":"ContainerDied","Data":"a31a13710a5b872579c942d240e7d08099809666582e485d3d947ed3bca396e8"}
Jul 17 10:06:35.886669 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:35.886582    2574 scope.go:117] "RemoveContainer" containerID="d3e2705dc1c2065928c845d3e2972ef95945c1ce3b25b642c896eb26156e66f6"
Jul 17 10:06:35.911054 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:35.911029    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpzd6"]
Jul 17 10:06:35.917117 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:35.917096    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-rpzd6"]
Jul 17 10:06:35.985861 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:35.985822    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-cfdp4"]
Jul 17 10:06:35.986370 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:35.986351    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b24c628-2cd0-4b36-a272-ebc5d3941287" containerName="revoke-keys"
Jul 17 10:06:35.986463 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:35.986373    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b24c628-2cd0-4b36-a272-ebc5d3941287" containerName="revoke-keys"
Jul 17 10:06:35.986463 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:35.986388    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2283bf9-6253-425f-b801-10fce8c6274e" containerName="revoke-keys"
Jul 17 10:06:35.986463 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:35.986397    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2283bf9-6253-425f-b801-10fce8c6274e" containerName="revoke-keys"
Jul 17 10:06:35.986463 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:35.986412    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2283bf9-6253-425f-b801-10fce8c6274e" containerName="revoke-keys"
Jul 17 10:06:35.986463 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:35.986421    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2283bf9-6253-425f-b801-10fce8c6274e" containerName="revoke-keys"
Jul 17 10:06:35.986463 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:35.986435    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b24c628-2cd0-4b36-a272-ebc5d3941287" containerName="revoke-keys"
Jul 17 10:06:35.986463 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:35.986443    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b24c628-2cd0-4b36-a272-ebc5d3941287" containerName="revoke-keys"
Jul 17 10:06:35.986463 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:35.986459    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2283bf9-6253-425f-b801-10fce8c6274e" containerName="revoke-keys"
Jul 17 10:06:35.986849 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:35.986467    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2283bf9-6253-425f-b801-10fce8c6274e" containerName="revoke-keys"
Jul 17 10:06:35.986849 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:35.986586    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2283bf9-6253-425f-b801-10fce8c6274e" containerName="revoke-keys"
Jul 17 10:06:35.986849 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:35.986599    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b24c628-2cd0-4b36-a272-ebc5d3941287" containerName="revoke-keys"
Jul 17 10:06:35.986849 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:35.986609    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2283bf9-6253-425f-b801-10fce8c6274e" containerName="revoke-keys"
Jul 17 10:06:35.986849 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:35.986620    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b24c628-2cd0-4b36-a272-ebc5d3941287" containerName="revoke-keys"
Jul 17 10:06:35.986849 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:35.986631    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2283bf9-6253-425f-b801-10fce8c6274e" containerName="revoke-keys"
Jul 17 10:06:36.000890 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:36.000852    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-cfdp4"
Jul 17 10:06:36.005414 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:36.005380    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-cfdp4"]
Jul 17 10:06:36.005914 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:36.005885    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:06:36.173160 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:36.173120    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37b434df-8874-4707-b2c3-64f3ba1d0e47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-cfdp4\" (UID: \"37b434df-8874-4707-b2c3-64f3ba1d0e47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-cfdp4"
Jul 17 10:06:36.274458 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:36.274426    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37b434df-8874-4707-b2c3-64f3ba1d0e47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-cfdp4\" (UID: \"37b434df-8874-4707-b2c3-64f3ba1d0e47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-cfdp4"
Jul 17 10:06:36.275083 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:36.275058    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37b434df-8874-4707-b2c3-64f3ba1d0e47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-cfdp4\" (UID: \"37b434df-8874-4707-b2c3-64f3ba1d0e47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-cfdp4"
Jul 17 10:06:36.315858 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:36.315828    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-cfdp4"
Jul 17 10:06:36.668367 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:36.668341    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-cfdp4"]
Jul 17 10:06:36.670288 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:06:36.670254    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37b434df_8874_4707_b2c3_64f3ba1d0e47.slice/crio-3e6b5877f62d442ac8d2416e671c9dab2a23ffb3b631ac4eb00ed5e1cc1fd483 WatchSource:0}: Error finding container 3e6b5877f62d442ac8d2416e671c9dab2a23ffb3b631ac4eb00ed5e1cc1fd483: Status 404 returned error can't find the container with id 3e6b5877f62d442ac8d2416e671c9dab2a23ffb3b631ac4eb00ed5e1cc1fd483
Jul 17 10:06:36.898373 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:36.898344    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2283bf9-6253-425f-b801-10fce8c6274e" path="/var/lib/kubelet/pods/f2283bf9-6253-425f-b801-10fce8c6274e/volumes"
Jul 17 10:06:36.898632 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:36.898616    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-cfdp4" event={"ID":"37b434df-8874-4707-b2c3-64f3ba1d0e47","Type":"ContainerStarted","Data":"3e6b5877f62d442ac8d2416e671c9dab2a23ffb3b631ac4eb00ed5e1cc1fd483"}
Jul 17 10:06:36.980678 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:36.980640    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt"]
Jul 17 10:06:36.981376 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:36.981347    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b24c628-2cd0-4b36-a272-ebc5d3941287" containerName="revoke-keys"
Jul 17 10:06:36.981376 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:36.981371    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b24c628-2cd0-4b36-a272-ebc5d3941287" containerName="revoke-keys"
Jul 17 10:06:36.981709 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:36.981485    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b24c628-2cd0-4b36-a272-ebc5d3941287" containerName="revoke-keys"
Jul 17 10:06:36.984614 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:36.984598    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt"
Jul 17 10:06:36.992953 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:36.992911    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt"]
Jul 17 10:06:37.183788 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:37.183698    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/613981d4-d1b5-4d16-96e4-4caff0744ea7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt\" (UID: \"613981d4-d1b5-4d16-96e4-4caff0744ea7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt"
Jul 17 10:06:37.284973 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:37.284940    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/613981d4-d1b5-4d16-96e4-4caff0744ea7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt\" (UID: \"613981d4-d1b5-4d16-96e4-4caff0744ea7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt"
Jul 17 10:06:37.285554 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:37.285533    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/613981d4-d1b5-4d16-96e4-4caff0744ea7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt\" (UID: \"613981d4-d1b5-4d16-96e4-4caff0744ea7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt"
Jul 17 10:06:37.295756 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:37.295730    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt"
Jul 17 10:06:37.433327 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:37.433299    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt"]
Jul 17 10:06:37.471159 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:06:37.471088    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod613981d4_d1b5_4d16_96e4_4caff0744ea7.slice/crio-1cb2ffe6ecd02b86c4b8d28157af9a8de2ebe64e2ca1cb2e581f184207c67e44 WatchSource:0}: Error finding container 1cb2ffe6ecd02b86c4b8d28157af9a8de2ebe64e2ca1cb2e581f184207c67e44: Status 404 returned error can't find the container with id 1cb2ffe6ecd02b86c4b8d28157af9a8de2ebe64e2ca1cb2e581f184207c67e44
Jul 17 10:06:37.901605 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:37.901564    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt" event={"ID":"613981d4-d1b5-4d16-96e4-4caff0744ea7","Type":"ContainerStarted","Data":"8c3c5ba4b8972a981e80c0fa547da0971e2c31b874d7fcba08c5140b8324b6a6"}
Jul 17 10:06:37.901776 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:37.901659    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt" event={"ID":"613981d4-d1b5-4d16-96e4-4caff0744ea7","Type":"ContainerStarted","Data":"1cb2ffe6ecd02b86c4b8d28157af9a8de2ebe64e2ca1cb2e581f184207c67e44"}
Jul 17 10:06:37.903205 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:37.903177    2574 generic.go:356] "Generic (PLEG): container finished" podID="37b434df-8874-4707-b2c3-64f3ba1d0e47" containerID="be9e54b95861198ac474af9b04f72fd69cea2bb72ae3dc6d49441bf45e850f03" exitCode=6
Jul 17 10:06:37.903333 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:37.903229    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-cfdp4" event={"ID":"37b434df-8874-4707-b2c3-64f3ba1d0e47","Type":"ContainerDied","Data":"be9e54b95861198ac474af9b04f72fd69cea2bb72ae3dc6d49441bf45e850f03"}
Jul 17 10:06:37.903434 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:37.903423    2574 scope.go:117] "RemoveContainer" containerID="be9e54b95861198ac474af9b04f72fd69cea2bb72ae3dc6d49441bf45e850f03"
Jul 17 10:06:37.918454 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:37.918401    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt" podStartSLOduration=1.918382572 podStartE2EDuration="1.918382572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:06:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:06:37.916447026 +0000 UTC m=+2395.499085426" watchObservedRunningTime="2026-07-17 10:06:37.918382572 +0000 UTC m=+2395.501020950"
Jul 17 10:06:38.909859 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:38.909826    2574 generic.go:356] "Generic (PLEG): container finished" podID="613981d4-d1b5-4d16-96e4-4caff0744ea7" containerID="8c3c5ba4b8972a981e80c0fa547da0971e2c31b874d7fcba08c5140b8324b6a6" exitCode=6
Jul 17 10:06:38.910347 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:38.909915    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt" event={"ID":"613981d4-d1b5-4d16-96e4-4caff0744ea7","Type":"ContainerDied","Data":"8c3c5ba4b8972a981e80c0fa547da0971e2c31b874d7fcba08c5140b8324b6a6"}
Jul 17 10:06:38.910347 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:38.910101    2574 scope.go:117] "RemoveContainer" containerID="8c3c5ba4b8972a981e80c0fa547da0971e2c31b874d7fcba08c5140b8324b6a6"
Jul 17 10:06:38.911894 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:38.911869    2574 generic.go:356] "Generic (PLEG): container finished" podID="37b434df-8874-4707-b2c3-64f3ba1d0e47" containerID="0643b2b79d7fa76a2599e25e28738830ea089bc60caa0b211573b8c5c617da2d" exitCode=6
Jul 17 10:06:38.911971 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:38.911946    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-cfdp4" event={"ID":"37b434df-8874-4707-b2c3-64f3ba1d0e47","Type":"ContainerDied","Data":"0643b2b79d7fa76a2599e25e28738830ea089bc60caa0b211573b8c5c617da2d"}
Jul 17 10:06:38.912020 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:38.912001    2574 scope.go:117] "RemoveContainer" containerID="be9e54b95861198ac474af9b04f72fd69cea2bb72ae3dc6d49441bf45e850f03"
Jul 17 10:06:38.912142 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:38.912128    2574 scope.go:117] "RemoveContainer" containerID="0643b2b79d7fa76a2599e25e28738830ea089bc60caa0b211573b8c5c617da2d"
Jul 17 10:06:38.912406 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:06:38.912326    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-cfdp4_odh-ai-gateway-infra(37b434df-8874-4707-b2c3-64f3ba1d0e47)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-cfdp4" podUID="37b434df-8874-4707-b2c3-64f3ba1d0e47"
Jul 17 10:06:39.919975 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:39.919936    2574 generic.go:356] "Generic (PLEG): container finished" podID="613981d4-d1b5-4d16-96e4-4caff0744ea7" containerID="72bc82e9f1df9d9c9c972afb0272bc3378795846cd652e29c3999464685a48c4" exitCode=6
Jul 17 10:06:39.920470 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:39.920019    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt" event={"ID":"613981d4-d1b5-4d16-96e4-4caff0744ea7","Type":"ContainerDied","Data":"72bc82e9f1df9d9c9c972afb0272bc3378795846cd652e29c3999464685a48c4"}
Jul 17 10:06:39.920470 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:39.920083    2574 scope.go:117] "RemoveContainer" containerID="8c3c5ba4b8972a981e80c0fa547da0971e2c31b874d7fcba08c5140b8324b6a6"
Jul 17 10:06:39.920470 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:39.920332    2574 scope.go:117] "RemoveContainer" containerID="72bc82e9f1df9d9c9c972afb0272bc3378795846cd652e29c3999464685a48c4"
Jul 17 10:06:39.920633 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:06:39.920572    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt_odh-ai-gateway-infra(613981d4-d1b5-4d16-96e4-4caff0744ea7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt" podUID="613981d4-d1b5-4d16-96e4-4caff0744ea7"
Jul 17 10:06:39.922079 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:39.922062    2574 scope.go:117] "RemoveContainer" containerID="0643b2b79d7fa76a2599e25e28738830ea089bc60caa0b211573b8c5c617da2d"
Jul 17 10:06:39.922277 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:06:39.922260    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-cfdp4_odh-ai-gateway-infra(37b434df-8874-4707-b2c3-64f3ba1d0e47)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-cfdp4" podUID="37b434df-8874-4707-b2c3-64f3ba1d0e47"
Jul 17 10:06:40.928805 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:40.928781    2574 scope.go:117] "RemoveContainer" containerID="72bc82e9f1df9d9c9c972afb0272bc3378795846cd652e29c3999464685a48c4"
Jul 17 10:06:40.929171 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:06:40.928988    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt_odh-ai-gateway-infra(613981d4-d1b5-4d16-96e4-4caff0744ea7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt" podUID="613981d4-d1b5-4d16-96e4-4caff0744ea7"
Jul 17 10:06:43.101528 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:43.101413    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dh564_84db06ab-87e2-43b3-82d0-13f06ba86d06/ovn-acl-logging/0.log"
Jul 17 10:06:43.134916 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:43.104189    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dh564_84db06ab-87e2-43b3-82d0-13f06ba86d06/ovn-acl-logging/0.log"
Jul 17 10:06:51.891620 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:51.891589    2574 scope.go:117] "RemoveContainer" containerID="0643b2b79d7fa76a2599e25e28738830ea089bc60caa0b211573b8c5c617da2d"
Jul 17 10:06:51.892257 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:51.891920    2574 scope.go:117] "RemoveContainer" containerID="72bc82e9f1df9d9c9c972afb0272bc3378795846cd652e29c3999464685a48c4"
Jul 17 10:06:52.996183 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:52.996146    2574 generic.go:356] "Generic (PLEG): container finished" podID="613981d4-d1b5-4d16-96e4-4caff0744ea7" containerID="0e67db91e8975cdd29d47a2a2cfa5dc42f2ae590cbf7c68793f472394125049b" exitCode=6
Jul 17 10:06:52.996712 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:52.996221    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt" event={"ID":"613981d4-d1b5-4d16-96e4-4caff0744ea7","Type":"ContainerDied","Data":"0e67db91e8975cdd29d47a2a2cfa5dc42f2ae590cbf7c68793f472394125049b"}
Jul 17 10:06:52.996712 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:52.996296    2574 scope.go:117] "RemoveContainer" containerID="72bc82e9f1df9d9c9c972afb0272bc3378795846cd652e29c3999464685a48c4"
Jul 17 10:06:52.996712 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:52.996654    2574 scope.go:117] "RemoveContainer" containerID="0e67db91e8975cdd29d47a2a2cfa5dc42f2ae590cbf7c68793f472394125049b"
Jul 17 10:06:52.996933 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:06:52.996906    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt_odh-ai-gateway-infra(613981d4-d1b5-4d16-96e4-4caff0744ea7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt" podUID="613981d4-d1b5-4d16-96e4-4caff0744ea7"
Jul 17 10:06:52.998341 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:52.998321    2574 generic.go:356] "Generic (PLEG): container finished" podID="37b434df-8874-4707-b2c3-64f3ba1d0e47" containerID="7a5a924f301ad6905e78ee56e82b78739b367d5504b7ee48b6ad9a5baa2fcb93" exitCode=6
Jul 17 10:06:52.998502 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:52.998374    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-cfdp4" event={"ID":"37b434df-8874-4707-b2c3-64f3ba1d0e47","Type":"ContainerDied","Data":"7a5a924f301ad6905e78ee56e82b78739b367d5504b7ee48b6ad9a5baa2fcb93"}
Jul 17 10:06:52.998604 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:52.998592    2574 scope.go:117] "RemoveContainer" containerID="7a5a924f301ad6905e78ee56e82b78739b367d5504b7ee48b6ad9a5baa2fcb93"
Jul 17 10:06:52.998800 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:06:52.998774    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-cfdp4_odh-ai-gateway-infra(37b434df-8874-4707-b2c3-64f3ba1d0e47)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-cfdp4" podUID="37b434df-8874-4707-b2c3-64f3ba1d0e47"
Jul 17 10:06:53.008114 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:53.008096    2574 scope.go:117] "RemoveContainer" containerID="0643b2b79d7fa76a2599e25e28738830ea089bc60caa0b211573b8c5c617da2d"
Jul 17 10:06:54.028758 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:54.028715    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt"]
Jul 17 10:06:54.045333 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:54.045302    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-cfdp4"]
Jul 17 10:06:54.182098 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:54.182074    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt"
Jul 17 10:06:54.206874 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:54.206850    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-cfdp4"
Jul 17 10:06:54.238741 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:54.238711    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37b434df-8874-4707-b2c3-64f3ba1d0e47-maas-api-service-ca\") pod \"37b434df-8874-4707-b2c3-64f3ba1d0e47\" (UID: \"37b434df-8874-4707-b2c3-64f3ba1d0e47\") "
Jul 17 10:06:54.238924 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:54.238809    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/613981d4-d1b5-4d16-96e4-4caff0744ea7-maas-api-service-ca\") pod \"613981d4-d1b5-4d16-96e4-4caff0744ea7\" (UID: \"613981d4-d1b5-4d16-96e4-4caff0744ea7\") "
Jul 17 10:06:54.239099 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:54.239076    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37b434df-8874-4707-b2c3-64f3ba1d0e47-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "37b434df-8874-4707-b2c3-64f3ba1d0e47" (UID: "37b434df-8874-4707-b2c3-64f3ba1d0e47"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:06:54.239165 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:54.239144    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/613981d4-d1b5-4d16-96e4-4caff0744ea7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "613981d4-d1b5-4d16-96e4-4caff0744ea7" (UID: "613981d4-d1b5-4d16-96e4-4caff0744ea7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:06:54.339939 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:54.339862    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37b434df-8874-4707-b2c3-64f3ba1d0e47-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:06:54.339939 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:54.339888    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/613981d4-d1b5-4d16-96e4-4caff0744ea7-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:06:55.014450 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.014420    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt"
Jul 17 10:06:55.014450 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.014435    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt" event={"ID":"613981d4-d1b5-4d16-96e4-4caff0744ea7","Type":"ContainerDied","Data":"1cb2ffe6ecd02b86c4b8d28157af9a8de2ebe64e2ca1cb2e581f184207c67e44"}
Jul 17 10:06:55.014702 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.014495    2574 scope.go:117] "RemoveContainer" containerID="0e67db91e8975cdd29d47a2a2cfa5dc42f2ae590cbf7c68793f472394125049b"
Jul 17 10:06:55.015933 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.015910    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-cfdp4"
Jul 17 10:06:55.016043 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.015955    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-cfdp4" event={"ID":"37b434df-8874-4707-b2c3-64f3ba1d0e47","Type":"ContainerDied","Data":"3e6b5877f62d442ac8d2416e671c9dab2a23ffb3b631ac4eb00ed5e1cc1fd483"}
Jul 17 10:06:55.023682 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.023666    2574 scope.go:117] "RemoveContainer" containerID="7a5a924f301ad6905e78ee56e82b78739b367d5504b7ee48b6ad9a5baa2fcb93"
Jul 17 10:06:55.033048 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.033027    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt"]
Jul 17 10:06:55.039586 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.039563    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-dd9nt"]
Jul 17 10:06:55.055360 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.055340    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-cfdp4"]
Jul 17 10:06:55.062793 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.062774    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-cfdp4"]
Jul 17 10:06:55.413884 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.413854    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-x694g"]
Jul 17 10:06:55.414313 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.414299    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37b434df-8874-4707-b2c3-64f3ba1d0e47" containerName="revoke-keys"
Jul 17 10:06:55.414372 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.414315    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="37b434df-8874-4707-b2c3-64f3ba1d0e47" containerName="revoke-keys"
Jul 17 10:06:55.414372 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.414328    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="613981d4-d1b5-4d16-96e4-4caff0744ea7" containerName="revoke-keys"
Jul 17 10:06:55.414372 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.414336    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="613981d4-d1b5-4d16-96e4-4caff0744ea7" containerName="revoke-keys"
Jul 17 10:06:55.414372 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.414344    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37b434df-8874-4707-b2c3-64f3ba1d0e47" containerName="revoke-keys"
Jul 17 10:06:55.414372 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.414352    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="37b434df-8874-4707-b2c3-64f3ba1d0e47" containerName="revoke-keys"
Jul 17 10:06:55.414372 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.414360    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="613981d4-d1b5-4d16-96e4-4caff0744ea7" containerName="revoke-keys"
Jul 17 10:06:55.414372 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.414365    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="613981d4-d1b5-4d16-96e4-4caff0744ea7" containerName="revoke-keys"
Jul 17 10:06:55.414609 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.414376    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37b434df-8874-4707-b2c3-64f3ba1d0e47" containerName="revoke-keys"
Jul 17 10:06:55.414609 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.414381    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="37b434df-8874-4707-b2c3-64f3ba1d0e47" containerName="revoke-keys"
Jul 17 10:06:55.414609 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.414447    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="613981d4-d1b5-4d16-96e4-4caff0744ea7" containerName="revoke-keys"
Jul 17 10:06:55.414609 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.414455    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="37b434df-8874-4707-b2c3-64f3ba1d0e47" containerName="revoke-keys"
Jul 17 10:06:55.414609 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.414460    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="37b434df-8874-4707-b2c3-64f3ba1d0e47" containerName="revoke-keys"
Jul 17 10:06:55.414609 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.414467    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="613981d4-d1b5-4d16-96e4-4caff0744ea7" containerName="revoke-keys"
Jul 17 10:06:55.414609 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.414472    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="37b434df-8874-4707-b2c3-64f3ba1d0e47" containerName="revoke-keys"
Jul 17 10:06:55.419109 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.419090    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-x694g"
Jul 17 10:06:55.421959 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.421938    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:06:55.425543 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.425521    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-x694g"]
Jul 17 10:06:55.552877 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.552844    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46d578f5-6f04-4b5e-aaca-8a510c5b5a61-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-x694g\" (UID: \"46d578f5-6f04-4b5e-aaca-8a510c5b5a61\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-x694g"
Jul 17 10:06:55.654053 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.654019    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46d578f5-6f04-4b5e-aaca-8a510c5b5a61-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-x694g\" (UID: \"46d578f5-6f04-4b5e-aaca-8a510c5b5a61\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-x694g"
Jul 17 10:06:55.654697 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.654678    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46d578f5-6f04-4b5e-aaca-8a510c5b5a61-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-x694g\" (UID: \"46d578f5-6f04-4b5e-aaca-8a510c5b5a61\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-x694g"
Jul 17 10:06:55.732384 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.732304    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-x694g"
Jul 17 10:06:55.874150 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:55.874114    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-x694g"]
Jul 17 10:06:55.874383 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:06:55.874358    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46d578f5_6f04_4b5e_aaca_8a510c5b5a61.slice/crio-1ec0c4af5aab7650e4a26a60f7a5bcd7cb7790a6f4a728fc57f4bc26cd98b378 WatchSource:0}: Error finding container 1ec0c4af5aab7650e4a26a60f7a5bcd7cb7790a6f4a728fc57f4bc26cd98b378: Status 404 returned error can't find the container with id 1ec0c4af5aab7650e4a26a60f7a5bcd7cb7790a6f4a728fc57f4bc26cd98b378
Jul 17 10:06:56.024695 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:56.024664    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-x694g" event={"ID":"46d578f5-6f04-4b5e-aaca-8a510c5b5a61","Type":"ContainerStarted","Data":"1ec0c4af5aab7650e4a26a60f7a5bcd7cb7790a6f4a728fc57f4bc26cd98b378"}
Jul 17 10:06:56.897980 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:56.897899    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37b434df-8874-4707-b2c3-64f3ba1d0e47" path="/var/lib/kubelet/pods/37b434df-8874-4707-b2c3-64f3ba1d0e47/volumes"
Jul 17 10:06:56.898370 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:56.898228    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="613981d4-d1b5-4d16-96e4-4caff0744ea7" path="/var/lib/kubelet/pods/613981d4-d1b5-4d16-96e4-4caff0744ea7/volumes"
Jul 17 10:06:57.031675 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:57.031634    2574 generic.go:356] "Generic (PLEG): container finished" podID="46d578f5-6f04-4b5e-aaca-8a510c5b5a61" containerID="19e61081248c72394800b4743a9fc74c4797c18260b0d65cd83786d053059214" exitCode=6
Jul 17 10:06:57.031675 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:57.031679    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-x694g" event={"ID":"46d578f5-6f04-4b5e-aaca-8a510c5b5a61","Type":"ContainerDied","Data":"19e61081248c72394800b4743a9fc74c4797c18260b0d65cd83786d053059214"}
Jul 17 10:06:57.031939 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:57.031900    2574 scope.go:117] "RemoveContainer" containerID="19e61081248c72394800b4743a9fc74c4797c18260b0d65cd83786d053059214"
Jul 17 10:06:58.038826 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:58.038796    2574 generic.go:356] "Generic (PLEG): container finished" podID="46d578f5-6f04-4b5e-aaca-8a510c5b5a61" containerID="2c0086e22146b3e1d103399dac18f7d65d2e5d8f40ab17b093c954d96921d32a" exitCode=6
Jul 17 10:06:58.039280 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:58.038847    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-x694g" event={"ID":"46d578f5-6f04-4b5e-aaca-8a510c5b5a61","Type":"ContainerDied","Data":"2c0086e22146b3e1d103399dac18f7d65d2e5d8f40ab17b093c954d96921d32a"}
Jul 17 10:06:58.039280 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:58.038902    2574 scope.go:117] "RemoveContainer" containerID="19e61081248c72394800b4743a9fc74c4797c18260b0d65cd83786d053059214"
Jul 17 10:06:58.039280 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:58.039033    2574 scope.go:117] "RemoveContainer" containerID="2c0086e22146b3e1d103399dac18f7d65d2e5d8f40ab17b093c954d96921d32a"
Jul 17 10:06:58.039418 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:06:58.039325    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-x694g_odh-ai-gateway-infra(46d578f5-6f04-4b5e-aaca-8a510c5b5a61)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-x694g" podUID="46d578f5-6f04-4b5e-aaca-8a510c5b5a61"
Jul 17 10:06:59.046200 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:06:59.046172    2574 scope.go:117] "RemoveContainer" containerID="2c0086e22146b3e1d103399dac18f7d65d2e5d8f40ab17b093c954d96921d32a"
Jul 17 10:06:59.046633 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:06:59.046391    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-x694g_odh-ai-gateway-infra(46d578f5-6f04-4b5e-aaca-8a510c5b5a61)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-x694g" podUID="46d578f5-6f04-4b5e-aaca-8a510c5b5a61"
Jul 17 10:07:08.137560 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:08.137472    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw"]
Jul 17 10:07:08.138045 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:08.138028    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="613981d4-d1b5-4d16-96e4-4caff0744ea7" containerName="revoke-keys"
Jul 17 10:07:08.138098 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:08.138051    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="613981d4-d1b5-4d16-96e4-4caff0744ea7" containerName="revoke-keys"
Jul 17 10:07:08.138131 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:08.138122    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="613981d4-d1b5-4d16-96e4-4caff0744ea7" containerName="revoke-keys"
Jul 17 10:07:08.142665 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:08.142648    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw"
Jul 17 10:07:08.152279 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:08.152233    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw"]
Jul 17 10:07:08.274365 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:08.274329    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45a2631a-9e41-4a52-93bc-bf3e841cbfb2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw\" (UID: \"45a2631a-9e41-4a52-93bc-bf3e841cbfb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw"
Jul 17 10:07:08.375146 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:08.375113    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45a2631a-9e41-4a52-93bc-bf3e841cbfb2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw\" (UID: \"45a2631a-9e41-4a52-93bc-bf3e841cbfb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw"
Jul 17 10:07:08.375704 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:08.375684    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45a2631a-9e41-4a52-93bc-bf3e841cbfb2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw\" (UID: \"45a2631a-9e41-4a52-93bc-bf3e841cbfb2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw"
Jul 17 10:07:08.454591 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:08.454554    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw"
Jul 17 10:07:08.599438 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:08.599414    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw"]
Jul 17 10:07:08.600999 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:07:08.600972    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45a2631a_9e41_4a52_93bc_bf3e841cbfb2.slice/crio-453eb01fdaf68146314d0b157cbcfc4520311fbb0b2d59f35a77bc14194574b9 WatchSource:0}: Error finding container 453eb01fdaf68146314d0b157cbcfc4520311fbb0b2d59f35a77bc14194574b9: Status 404 returned error can't find the container with id 453eb01fdaf68146314d0b157cbcfc4520311fbb0b2d59f35a77bc14194574b9
Jul 17 10:07:09.103318 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:09.103216    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw" event={"ID":"45a2631a-9e41-4a52-93bc-bf3e841cbfb2","Type":"ContainerStarted","Data":"0bc71b0e3efe9076605614a20661ad45a8f847abdeab57942158caac29ade0d1"}
Jul 17 10:07:09.103318 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:09.103298    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw" event={"ID":"45a2631a-9e41-4a52-93bc-bf3e841cbfb2","Type":"ContainerStarted","Data":"453eb01fdaf68146314d0b157cbcfc4520311fbb0b2d59f35a77bc14194574b9"}
Jul 17 10:07:09.128602 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:09.128559    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw" podStartSLOduration=1.128543592 podStartE2EDuration="1.128543592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:07:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:07:09.127027709 +0000 UTC m=+2426.709666096" watchObservedRunningTime="2026-07-17 10:07:09.128543592 +0000 UTC m=+2426.711181970"
Jul 17 10:07:10.109561 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:10.109528    2574 generic.go:356] "Generic (PLEG): container finished" podID="45a2631a-9e41-4a52-93bc-bf3e841cbfb2" containerID="0bc71b0e3efe9076605614a20661ad45a8f847abdeab57942158caac29ade0d1" exitCode=6
Jul 17 10:07:10.109992 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:10.109618    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw" event={"ID":"45a2631a-9e41-4a52-93bc-bf3e841cbfb2","Type":"ContainerDied","Data":"0bc71b0e3efe9076605614a20661ad45a8f847abdeab57942158caac29ade0d1"}
Jul 17 10:07:10.109992 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:10.109807    2574 scope.go:117] "RemoveContainer" containerID="0bc71b0e3efe9076605614a20661ad45a8f847abdeab57942158caac29ade0d1"
Jul 17 10:07:11.117184 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:11.117101    2574 generic.go:356] "Generic (PLEG): container finished" podID="45a2631a-9e41-4a52-93bc-bf3e841cbfb2" containerID="efc108da94199680af480509668941429d8180535386582e5798f56755e67e78" exitCode=6
Jul 17 10:07:11.117615 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:11.117183    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw" event={"ID":"45a2631a-9e41-4a52-93bc-bf3e841cbfb2","Type":"ContainerDied","Data":"efc108da94199680af480509668941429d8180535386582e5798f56755e67e78"}
Jul 17 10:07:11.117615 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:11.117270    2574 scope.go:117] "RemoveContainer" containerID="0bc71b0e3efe9076605614a20661ad45a8f847abdeab57942158caac29ade0d1"
Jul 17 10:07:11.117615 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:11.117388    2574 scope.go:117] "RemoveContainer" containerID="efc108da94199680af480509668941429d8180535386582e5798f56755e67e78"
Jul 17 10:07:11.117615 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:07:11.117588    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw_odh-ai-gateway-infra(45a2631a-9e41-4a52-93bc-bf3e841cbfb2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw" podUID="45a2631a-9e41-4a52-93bc-bf3e841cbfb2"
Jul 17 10:07:11.891428 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:11.891393    2574 scope.go:117] "RemoveContainer" containerID="2c0086e22146b3e1d103399dac18f7d65d2e5d8f40ab17b093c954d96921d32a"
Jul 17 10:07:12.124376 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:12.124344    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-x694g" event={"ID":"46d578f5-6f04-4b5e-aaca-8a510c5b5a61","Type":"ContainerStarted","Data":"904fcfbae6f6f1c1559927ff76497d04f24466cabb6383b4022ae4c20dd42319"}
Jul 17 10:07:12.126175 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:12.126158    2574 scope.go:117] "RemoveContainer" containerID="efc108da94199680af480509668941429d8180535386582e5798f56755e67e78"
Jul 17 10:07:12.126450 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:07:12.126429    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw_odh-ai-gateway-infra(45a2631a-9e41-4a52-93bc-bf3e841cbfb2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw" podUID="45a2631a-9e41-4a52-93bc-bf3e841cbfb2"
Jul 17 10:07:12.143230 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:12.143157    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-x694g" podStartSLOduration=17.14314283 podStartE2EDuration="17.14314283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:06:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:07:12.141685284 +0000 UTC m=+2429.724323681" watchObservedRunningTime="2026-07-17 10:07:12.14314283 +0000 UTC m=+2429.725781206"
Jul 17 10:07:13.133132 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:13.133092    2574 generic.go:356] "Generic (PLEG): container finished" podID="46d578f5-6f04-4b5e-aaca-8a510c5b5a61" containerID="904fcfbae6f6f1c1559927ff76497d04f24466cabb6383b4022ae4c20dd42319" exitCode=6
Jul 17 10:07:13.133564 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:13.133166    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-x694g" event={"ID":"46d578f5-6f04-4b5e-aaca-8a510c5b5a61","Type":"ContainerDied","Data":"904fcfbae6f6f1c1559927ff76497d04f24466cabb6383b4022ae4c20dd42319"}
Jul 17 10:07:13.133564 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:13.133229    2574 scope.go:117] "RemoveContainer" containerID="2c0086e22146b3e1d103399dac18f7d65d2e5d8f40ab17b093c954d96921d32a"
Jul 17 10:07:13.133564 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:13.133509    2574 scope.go:117] "RemoveContainer" containerID="904fcfbae6f6f1c1559927ff76497d04f24466cabb6383b4022ae4c20dd42319"
Jul 17 10:07:13.133781 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:07:13.133763    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-x694g_odh-ai-gateway-infra(46d578f5-6f04-4b5e-aaca-8a510c5b5a61)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-x694g" podUID="46d578f5-6f04-4b5e-aaca-8a510c5b5a61"
Jul 17 10:07:13.153127 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:13.153104    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-x694g"]
Jul 17 10:07:14.283624 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:14.283601    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-x694g"
Jul 17 10:07:14.438829 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:14.438787    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46d578f5-6f04-4b5e-aaca-8a510c5b5a61-maas-api-service-ca\") pod \"46d578f5-6f04-4b5e-aaca-8a510c5b5a61\" (UID: \"46d578f5-6f04-4b5e-aaca-8a510c5b5a61\") "
Jul 17 10:07:14.439141 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:14.439119    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46d578f5-6f04-4b5e-aaca-8a510c5b5a61-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "46d578f5-6f04-4b5e-aaca-8a510c5b5a61" (UID: "46d578f5-6f04-4b5e-aaca-8a510c5b5a61"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:07:14.540127 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:14.540095    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/46d578f5-6f04-4b5e-aaca-8a510c5b5a61-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:07:15.146498 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:15.146465    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-x694g"
Jul 17 10:07:15.146682 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:15.146493    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-x694g" event={"ID":"46d578f5-6f04-4b5e-aaca-8a510c5b5a61","Type":"ContainerDied","Data":"1ec0c4af5aab7650e4a26a60f7a5bcd7cb7790a6f4a728fc57f4bc26cd98b378"}
Jul 17 10:07:15.146682 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:15.146562    2574 scope.go:117] "RemoveContainer" containerID="904fcfbae6f6f1c1559927ff76497d04f24466cabb6383b4022ae4c20dd42319"
Jul 17 10:07:15.164840 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:15.164778    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-x694g"]
Jul 17 10:07:15.171216 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:15.171192    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-x694g"]
Jul 17 10:07:16.898104 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:16.898073    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46d578f5-6f04-4b5e-aaca-8a510c5b5a61" path="/var/lib/kubelet/pods/46d578f5-6f04-4b5e-aaca-8a510c5b5a61/volumes"
Jul 17 10:07:22.894542 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:22.894510    2574 scope.go:117] "RemoveContainer" containerID="efc108da94199680af480509668941429d8180535386582e5798f56755e67e78"
Jul 17 10:07:24.198549 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:24.198518    2574 generic.go:356] "Generic (PLEG): container finished" podID="45a2631a-9e41-4a52-93bc-bf3e841cbfb2" containerID="3d3ec97e4baccf174a3e88bd9afd098652be3709dc5defaca2254b7ca3bf9d2c" exitCode=6
Jul 17 10:07:24.199072 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:24.198571    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw" event={"ID":"45a2631a-9e41-4a52-93bc-bf3e841cbfb2","Type":"ContainerDied","Data":"3d3ec97e4baccf174a3e88bd9afd098652be3709dc5defaca2254b7ca3bf9d2c"}
Jul 17 10:07:24.199072 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:24.198623    2574 scope.go:117] "RemoveContainer" containerID="efc108da94199680af480509668941429d8180535386582e5798f56755e67e78"
Jul 17 10:07:24.199072 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:24.198971    2574 scope.go:117] "RemoveContainer" containerID="3d3ec97e4baccf174a3e88bd9afd098652be3709dc5defaca2254b7ca3bf9d2c"
Jul 17 10:07:24.199298 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:07:24.199277    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw_odh-ai-gateway-infra(45a2631a-9e41-4a52-93bc-bf3e841cbfb2)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw" podUID="45a2631a-9e41-4a52-93bc-bf3e841cbfb2"
Jul 17 10:07:25.228084 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:25.228051    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw"]
Jul 17 10:07:25.370319 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:25.370295    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw"
Jul 17 10:07:25.438057 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:25.438028    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45a2631a-9e41-4a52-93bc-bf3e841cbfb2-maas-api-service-ca\") pod \"45a2631a-9e41-4a52-93bc-bf3e841cbfb2\" (UID: \"45a2631a-9e41-4a52-93bc-bf3e841cbfb2\") "
Jul 17 10:07:25.438391 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:25.438369    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45a2631a-9e41-4a52-93bc-bf3e841cbfb2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "45a2631a-9e41-4a52-93bc-bf3e841cbfb2" (UID: "45a2631a-9e41-4a52-93bc-bf3e841cbfb2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:07:25.539351 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:25.539235    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/45a2631a-9e41-4a52-93bc-bf3e841cbfb2-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:07:26.034993 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:26.034960    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd"]
Jul 17 10:07:26.035399 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:26.035386    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46d578f5-6f04-4b5e-aaca-8a510c5b5a61" containerName="revoke-keys"
Jul 17 10:07:26.035457 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:26.035400    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="46d578f5-6f04-4b5e-aaca-8a510c5b5a61" containerName="revoke-keys"
Jul 17 10:07:26.035457 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:26.035409    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45a2631a-9e41-4a52-93bc-bf3e841cbfb2" containerName="revoke-keys"
Jul 17 10:07:26.035457 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:26.035430    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="45a2631a-9e41-4a52-93bc-bf3e841cbfb2" containerName="revoke-keys"
Jul 17 10:07:26.035457 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:26.035454    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46d578f5-6f04-4b5e-aaca-8a510c5b5a61" containerName="revoke-keys"
Jul 17 10:07:26.035658 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:26.035459    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="46d578f5-6f04-4b5e-aaca-8a510c5b5a61" containerName="revoke-keys"
Jul 17 10:07:26.035658 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:26.035470    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45a2631a-9e41-4a52-93bc-bf3e841cbfb2" containerName="revoke-keys"
Jul 17 10:07:26.035658 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:26.035475    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="45a2631a-9e41-4a52-93bc-bf3e841cbfb2" containerName="revoke-keys"
Jul 17 10:07:26.035658 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:26.035526    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="46d578f5-6f04-4b5e-aaca-8a510c5b5a61" containerName="revoke-keys"
Jul 17 10:07:26.035658 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:26.035534    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="45a2631a-9e41-4a52-93bc-bf3e841cbfb2" containerName="revoke-keys"
Jul 17 10:07:26.035658 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:26.035542    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="46d578f5-6f04-4b5e-aaca-8a510c5b5a61" containerName="revoke-keys"
Jul 17 10:07:26.035658 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:26.035548    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="45a2631a-9e41-4a52-93bc-bf3e841cbfb2" containerName="revoke-keys"
Jul 17 10:07:26.035658 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:26.035555    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="45a2631a-9e41-4a52-93bc-bf3e841cbfb2" containerName="revoke-keys"
Jul 17 10:07:26.038834 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:26.038819    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd"
Jul 17 10:07:26.048589 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:26.048561    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd"]
Jul 17 10:07:26.145495 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:26.145451    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8e825ec-7600-4d37-b262-ecba83d00136-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd\" (UID: \"d8e825ec-7600-4d37-b262-ecba83d00136\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd"
Jul 17 10:07:26.213255 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:26.213223    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw"
Jul 17 10:07:26.213429 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:26.213272    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw" event={"ID":"45a2631a-9e41-4a52-93bc-bf3e841cbfb2","Type":"ContainerDied","Data":"453eb01fdaf68146314d0b157cbcfc4520311fbb0b2d59f35a77bc14194574b9"}
Jul 17 10:07:26.213429 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:26.213358    2574 scope.go:117] "RemoveContainer" containerID="3d3ec97e4baccf174a3e88bd9afd098652be3709dc5defaca2254b7ca3bf9d2c"
Jul 17 10:07:26.235986 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:26.235958    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw"]
Jul 17 10:07:26.242020 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:26.241997    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-7tgmw"]
Jul 17 10:07:26.246218 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:26.246197    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8e825ec-7600-4d37-b262-ecba83d00136-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd\" (UID: \"d8e825ec-7600-4d37-b262-ecba83d00136\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd"
Jul 17 10:07:26.246766 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:26.246748    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8e825ec-7600-4d37-b262-ecba83d00136-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd\" (UID: \"d8e825ec-7600-4d37-b262-ecba83d00136\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd"
Jul 17 10:07:26.351145 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:26.351080    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd"
Jul 17 10:07:26.490941 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:26.490915    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd"]
Jul 17 10:07:26.493542 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:07:26.493513    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8e825ec_7600_4d37_b262_ecba83d00136.slice/crio-3c231aebff98f17bb6802ab893d030a707dfc50eebed68777f0252cd0755e28c WatchSource:0}: Error finding container 3c231aebff98f17bb6802ab893d030a707dfc50eebed68777f0252cd0755e28c: Status 404 returned error can't find the container with id 3c231aebff98f17bb6802ab893d030a707dfc50eebed68777f0252cd0755e28c
Jul 17 10:07:26.898472 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:26.898436    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45a2631a-9e41-4a52-93bc-bf3e841cbfb2" path="/var/lib/kubelet/pods/45a2631a-9e41-4a52-93bc-bf3e841cbfb2/volumes"
Jul 17 10:07:27.018749 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:27.018669    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5"]
Jul 17 10:07:27.019110 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:27.019098    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46d578f5-6f04-4b5e-aaca-8a510c5b5a61" containerName="revoke-keys"
Jul 17 10:07:27.019156 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:27.019112    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="46d578f5-6f04-4b5e-aaca-8a510c5b5a61" containerName="revoke-keys"
Jul 17 10:07:27.019156 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:27.019131    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45a2631a-9e41-4a52-93bc-bf3e841cbfb2" containerName="revoke-keys"
Jul 17 10:07:27.019156 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:27.019137    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="45a2631a-9e41-4a52-93bc-bf3e841cbfb2" containerName="revoke-keys"
Jul 17 10:07:27.019269 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:27.019210    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="46d578f5-6f04-4b5e-aaca-8a510c5b5a61" containerName="revoke-keys"
Jul 17 10:07:27.022504 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:27.022485    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5"
Jul 17 10:07:27.029762 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:27.029739    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5"]
Jul 17 10:07:27.053980 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:27.053954    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49e09a7a-1ebf-4d60-8408-2aab7c42eea5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5\" (UID: \"49e09a7a-1ebf-4d60-8408-2aab7c42eea5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5"
Jul 17 10:07:27.155569 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:27.155537    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49e09a7a-1ebf-4d60-8408-2aab7c42eea5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5\" (UID: \"49e09a7a-1ebf-4d60-8408-2aab7c42eea5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5"
Jul 17 10:07:27.156111 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:27.156091    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49e09a7a-1ebf-4d60-8408-2aab7c42eea5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5\" (UID: \"49e09a7a-1ebf-4d60-8408-2aab7c42eea5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5"
Jul 17 10:07:27.220873 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:27.220840    2574 generic.go:356] "Generic (PLEG): container finished" podID="d8e825ec-7600-4d37-b262-ecba83d00136" containerID="74923f7b53a090bf26711bc4bc12eb76a6beeec33d936f02fed240b0e9c837c4" exitCode=6
Jul 17 10:07:27.221048 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:27.220882    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd" event={"ID":"d8e825ec-7600-4d37-b262-ecba83d00136","Type":"ContainerDied","Data":"74923f7b53a090bf26711bc4bc12eb76a6beeec33d936f02fed240b0e9c837c4"}
Jul 17 10:07:27.221048 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:27.220917    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd" event={"ID":"d8e825ec-7600-4d37-b262-ecba83d00136","Type":"ContainerStarted","Data":"3c231aebff98f17bb6802ab893d030a707dfc50eebed68777f0252cd0755e28c"}
Jul 17 10:07:27.221166 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:27.221120    2574 scope.go:117] "RemoveContainer" containerID="74923f7b53a090bf26711bc4bc12eb76a6beeec33d936f02fed240b0e9c837c4"
Jul 17 10:07:27.335791 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:27.335750    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5"
Jul 17 10:07:27.472516 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:27.472493    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5"]
Jul 17 10:07:27.474536 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:07:27.474511    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49e09a7a_1ebf_4d60_8408_2aab7c42eea5.slice/crio-5c00c584cb439585f739640334ff0a5454e0108c1f92ef5e97a5fab7c4dd08ac WatchSource:0}: Error finding container 5c00c584cb439585f739640334ff0a5454e0108c1f92ef5e97a5fab7c4dd08ac: Status 404 returned error can't find the container with id 5c00c584cb439585f739640334ff0a5454e0108c1f92ef5e97a5fab7c4dd08ac
Jul 17 10:07:28.228087 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:28.228057    2574 generic.go:356] "Generic (PLEG): container finished" podID="49e09a7a-1ebf-4d60-8408-2aab7c42eea5" containerID="49875786539cb77779a8b3ccbc46c0100df434586c26d27b3de34a88d25f5014" exitCode=6
Jul 17 10:07:28.228234 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:28.228137    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5" event={"ID":"49e09a7a-1ebf-4d60-8408-2aab7c42eea5","Type":"ContainerDied","Data":"49875786539cb77779a8b3ccbc46c0100df434586c26d27b3de34a88d25f5014"}
Jul 17 10:07:28.228234 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:28.228195    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5" event={"ID":"49e09a7a-1ebf-4d60-8408-2aab7c42eea5","Type":"ContainerStarted","Data":"5c00c584cb439585f739640334ff0a5454e0108c1f92ef5e97a5fab7c4dd08ac"}
Jul 17 10:07:28.228377 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:28.228319    2574 scope.go:117] "RemoveContainer" containerID="49875786539cb77779a8b3ccbc46c0100df434586c26d27b3de34a88d25f5014"
Jul 17 10:07:28.230047 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:28.230027    2574 generic.go:356] "Generic (PLEG): container finished" podID="d8e825ec-7600-4d37-b262-ecba83d00136" containerID="0000c171ce141772ada1f2fcfc253b074e07bea93570529dceb078b72936ca4d" exitCode=6
Jul 17 10:07:28.230152 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:28.230106    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd" event={"ID":"d8e825ec-7600-4d37-b262-ecba83d00136","Type":"ContainerDied","Data":"0000c171ce141772ada1f2fcfc253b074e07bea93570529dceb078b72936ca4d"}
Jul 17 10:07:28.230222 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:28.230158    2574 scope.go:117] "RemoveContainer" containerID="74923f7b53a090bf26711bc4bc12eb76a6beeec33d936f02fed240b0e9c837c4"
Jul 17 10:07:28.230370 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:28.230348    2574 scope.go:117] "RemoveContainer" containerID="0000c171ce141772ada1f2fcfc253b074e07bea93570529dceb078b72936ca4d"
Jul 17 10:07:28.230637 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:07:28.230598    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd_odh-ai-gateway-infra(d8e825ec-7600-4d37-b262-ecba83d00136)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd" podUID="d8e825ec-7600-4d37-b262-ecba83d00136"
Jul 17 10:07:29.238171 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:29.238137    2574 scope.go:117] "RemoveContainer" containerID="0000c171ce141772ada1f2fcfc253b074e07bea93570529dceb078b72936ca4d"
Jul 17 10:07:29.238670 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:07:29.238425    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd_odh-ai-gateway-infra(d8e825ec-7600-4d37-b262-ecba83d00136)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd" podUID="d8e825ec-7600-4d37-b262-ecba83d00136"
Jul 17 10:07:29.239989 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:29.239964    2574 generic.go:356] "Generic (PLEG): container finished" podID="49e09a7a-1ebf-4d60-8408-2aab7c42eea5" containerID="bfd2552f0e1820ac3ecf7cb0fe081759a830af83bd8a2aed83c11e2169feba9e" exitCode=6
Jul 17 10:07:29.240115 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:29.239997    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5" event={"ID":"49e09a7a-1ebf-4d60-8408-2aab7c42eea5","Type":"ContainerDied","Data":"bfd2552f0e1820ac3ecf7cb0fe081759a830af83bd8a2aed83c11e2169feba9e"}
Jul 17 10:07:29.240115 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:29.240038    2574 scope.go:117] "RemoveContainer" containerID="49875786539cb77779a8b3ccbc46c0100df434586c26d27b3de34a88d25f5014"
Jul 17 10:07:29.240253 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:29.240222    2574 scope.go:117] "RemoveContainer" containerID="bfd2552f0e1820ac3ecf7cb0fe081759a830af83bd8a2aed83c11e2169feba9e"
Jul 17 10:07:29.240534 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:07:29.240510    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5_odh-ai-gateway-infra(49e09a7a-1ebf-4d60-8408-2aab7c42eea5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5" podUID="49e09a7a-1ebf-4d60-8408-2aab7c42eea5"
Jul 17 10:07:30.247968 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:30.247946    2574 scope.go:117] "RemoveContainer" containerID="bfd2552f0e1820ac3ecf7cb0fe081759a830af83bd8a2aed83c11e2169feba9e"
Jul 17 10:07:30.248331 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:07:30.248126    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5_odh-ai-gateway-infra(49e09a7a-1ebf-4d60-8408-2aab7c42eea5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5" podUID="49e09a7a-1ebf-4d60-8408-2aab7c42eea5"
Jul 17 10:07:40.891494 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:40.891458    2574 scope.go:117] "RemoveContainer" containerID="bfd2552f0e1820ac3ecf7cb0fe081759a830af83bd8a2aed83c11e2169feba9e"
Jul 17 10:07:41.308927 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:41.308888    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5" event={"ID":"49e09a7a-1ebf-4d60-8408-2aab7c42eea5","Type":"ContainerStarted","Data":"fe228997bebddaa90a127e3c4d1e77fefcd361cb4d9c0e77dbe217c5d0be575d"}
Jul 17 10:07:41.333193 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:41.333141    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5" podStartSLOduration=14.333125654 podStartE2EDuration="14.333125654s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:07:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:07:41.329759119 +0000 UTC m=+2458.912397506" watchObservedRunningTime="2026-07-17 10:07:41.333125654 +0000 UTC m=+2458.915764030"
Jul 17 10:07:41.892014 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:41.891973    2574 scope.go:117] "RemoveContainer" containerID="0000c171ce141772ada1f2fcfc253b074e07bea93570529dceb078b72936ca4d"
Jul 17 10:07:42.316978 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:42.316943    2574 generic.go:356] "Generic (PLEG): container finished" podID="49e09a7a-1ebf-4d60-8408-2aab7c42eea5" containerID="fe228997bebddaa90a127e3c4d1e77fefcd361cb4d9c0e77dbe217c5d0be575d" exitCode=6
Jul 17 10:07:42.317150 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:42.317020    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5" event={"ID":"49e09a7a-1ebf-4d60-8408-2aab7c42eea5","Type":"ContainerDied","Data":"fe228997bebddaa90a127e3c4d1e77fefcd361cb4d9c0e77dbe217c5d0be575d"}
Jul 17 10:07:42.317150 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:42.317078    2574 scope.go:117] "RemoveContainer" containerID="bfd2552f0e1820ac3ecf7cb0fe081759a830af83bd8a2aed83c11e2169feba9e"
Jul 17 10:07:42.317447 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:42.317429    2574 scope.go:117] "RemoveContainer" containerID="fe228997bebddaa90a127e3c4d1e77fefcd361cb4d9c0e77dbe217c5d0be575d"
Jul 17 10:07:42.317715 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:07:42.317687    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5_odh-ai-gateway-infra(49e09a7a-1ebf-4d60-8408-2aab7c42eea5)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5" podUID="49e09a7a-1ebf-4d60-8408-2aab7c42eea5"
Jul 17 10:07:42.319444 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:42.319412    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd" event={"ID":"d8e825ec-7600-4d37-b262-ecba83d00136","Type":"ContainerStarted","Data":"2543d3f56e135f537c999c7bfd6a8bfa2f51d4688a0652f3a49e00b0382956c0"}
Jul 17 10:07:42.369757 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:42.369715    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5"]
Jul 17 10:07:42.427512 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:42.427453    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd" podStartSLOduration=16.427435999 podStartE2EDuration="16.427435999s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:07:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:07:42.424713861 +0000 UTC m=+2460.007352246" watchObservedRunningTime="2026-07-17 10:07:42.427435999 +0000 UTC m=+2460.010074375"
Jul 17 10:07:43.326501 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:43.326464    2574 generic.go:356] "Generic (PLEG): container finished" podID="d8e825ec-7600-4d37-b262-ecba83d00136" containerID="2543d3f56e135f537c999c7bfd6a8bfa2f51d4688a0652f3a49e00b0382956c0" exitCode=6
Jul 17 10:07:43.326900 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:43.326537    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd" event={"ID":"d8e825ec-7600-4d37-b262-ecba83d00136","Type":"ContainerDied","Data":"2543d3f56e135f537c999c7bfd6a8bfa2f51d4688a0652f3a49e00b0382956c0"}
Jul 17 10:07:43.326900 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:43.326598    2574 scope.go:117] "RemoveContainer" containerID="0000c171ce141772ada1f2fcfc253b074e07bea93570529dceb078b72936ca4d"
Jul 17 10:07:43.326900 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:43.326878    2574 scope.go:117] "RemoveContainer" containerID="2543d3f56e135f537c999c7bfd6a8bfa2f51d4688a0652f3a49e00b0382956c0"
Jul 17 10:07:43.327120 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:07:43.327095    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd_odh-ai-gateway-infra(d8e825ec-7600-4d37-b262-ecba83d00136)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd" podUID="d8e825ec-7600-4d37-b262-ecba83d00136"
Jul 17 10:07:43.445220 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:43.445190    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd"]
Jul 17 10:07:43.467318 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:43.467288    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5"
Jul 17 10:07:43.617810 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:43.617722    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49e09a7a-1ebf-4d60-8408-2aab7c42eea5-maas-api-service-ca\") pod \"49e09a7a-1ebf-4d60-8408-2aab7c42eea5\" (UID: \"49e09a7a-1ebf-4d60-8408-2aab7c42eea5\") "
Jul 17 10:07:43.618078 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:43.618053    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49e09a7a-1ebf-4d60-8408-2aab7c42eea5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "49e09a7a-1ebf-4d60-8408-2aab7c42eea5" (UID: "49e09a7a-1ebf-4d60-8408-2aab7c42eea5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:07:43.718755 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:43.718718    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/49e09a7a-1ebf-4d60-8408-2aab7c42eea5-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:07:44.334812 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:44.334784    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5"
Jul 17 10:07:44.335348 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:44.334778    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5" event={"ID":"49e09a7a-1ebf-4d60-8408-2aab7c42eea5","Type":"ContainerDied","Data":"5c00c584cb439585f739640334ff0a5454e0108c1f92ef5e97a5fab7c4dd08ac"}
Jul 17 10:07:44.335348 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:44.334899    2574 scope.go:117] "RemoveContainer" containerID="fe228997bebddaa90a127e3c4d1e77fefcd361cb4d9c0e77dbe217c5d0be575d"
Jul 17 10:07:44.369600 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:44.369566    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5"]
Jul 17 10:07:44.373438 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:44.373413    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-wl8k5"]
Jul 17 10:07:44.484084 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:44.484061    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd"
Jul 17 10:07:44.626760 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:44.626664    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8e825ec-7600-4d37-b262-ecba83d00136-maas-api-service-ca\") pod \"d8e825ec-7600-4d37-b262-ecba83d00136\" (UID: \"d8e825ec-7600-4d37-b262-ecba83d00136\") "
Jul 17 10:07:44.627012 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:44.626987    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8e825ec-7600-4d37-b262-ecba83d00136-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d8e825ec-7600-4d37-b262-ecba83d00136" (UID: "d8e825ec-7600-4d37-b262-ecba83d00136"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:07:44.728265 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:44.728202    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d8e825ec-7600-4d37-b262-ecba83d00136-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:07:44.898421 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:44.898340    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49e09a7a-1ebf-4d60-8408-2aab7c42eea5" path="/var/lib/kubelet/pods/49e09a7a-1ebf-4d60-8408-2aab7c42eea5/volumes"
Jul 17 10:07:45.344652 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.344617    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd" event={"ID":"d8e825ec-7600-4d37-b262-ecba83d00136","Type":"ContainerDied","Data":"3c231aebff98f17bb6802ab893d030a707dfc50eebed68777f0252cd0755e28c"}
Jul 17 10:07:45.345115 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.344680    2574 scope.go:117] "RemoveContainer" containerID="2543d3f56e135f537c999c7bfd6a8bfa2f51d4688a0652f3a49e00b0382956c0"
Jul 17 10:07:45.345115 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.344631    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd"
Jul 17 10:07:45.364067 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.364040    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd"]
Jul 17 10:07:45.369412 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.369377    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-8v6vd"]
Jul 17 10:07:45.463178 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.463141    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-f22wc"]
Jul 17 10:07:45.463795 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.463778    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8e825ec-7600-4d37-b262-ecba83d00136" containerName="revoke-keys"
Jul 17 10:07:45.463840 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.463799    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8e825ec-7600-4d37-b262-ecba83d00136" containerName="revoke-keys"
Jul 17 10:07:45.463840 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.463814    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49e09a7a-1ebf-4d60-8408-2aab7c42eea5" containerName="revoke-keys"
Jul 17 10:07:45.463840 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.463820    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="49e09a7a-1ebf-4d60-8408-2aab7c42eea5" containerName="revoke-keys"
Jul 17 10:07:45.463840 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.463828    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49e09a7a-1ebf-4d60-8408-2aab7c42eea5" containerName="revoke-keys"
Jul 17 10:07:45.463840 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.463834    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="49e09a7a-1ebf-4d60-8408-2aab7c42eea5" containerName="revoke-keys"
Jul 17 10:07:45.464001 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.463846    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8e825ec-7600-4d37-b262-ecba83d00136" containerName="revoke-keys"
Jul 17 10:07:45.464001 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.463852    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8e825ec-7600-4d37-b262-ecba83d00136" containerName="revoke-keys"
Jul 17 10:07:45.464001 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.463859    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49e09a7a-1ebf-4d60-8408-2aab7c42eea5" containerName="revoke-keys"
Jul 17 10:07:45.464001 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.463865    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="49e09a7a-1ebf-4d60-8408-2aab7c42eea5" containerName="revoke-keys"
Jul 17 10:07:45.464001 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.463871    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8e825ec-7600-4d37-b262-ecba83d00136" containerName="revoke-keys"
Jul 17 10:07:45.464001 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.463877    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8e825ec-7600-4d37-b262-ecba83d00136" containerName="revoke-keys"
Jul 17 10:07:45.464001 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.463938    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="49e09a7a-1ebf-4d60-8408-2aab7c42eea5" containerName="revoke-keys"
Jul 17 10:07:45.464001 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.463946    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="49e09a7a-1ebf-4d60-8408-2aab7c42eea5" containerName="revoke-keys"
Jul 17 10:07:45.464001 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.463954    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8e825ec-7600-4d37-b262-ecba83d00136" containerName="revoke-keys"
Jul 17 10:07:45.464001 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.463959    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8e825ec-7600-4d37-b262-ecba83d00136" containerName="revoke-keys"
Jul 17 10:07:45.464001 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.463965    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8e825ec-7600-4d37-b262-ecba83d00136" containerName="revoke-keys"
Jul 17 10:07:45.468555 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.468538    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-f22wc"
Jul 17 10:07:45.472409 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.472393    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:07:45.476724 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.476409    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-f22wc"]
Jul 17 10:07:45.535101 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.535061    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19f5b515-a5a6-42e0-84fb-f77520389f2b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-f22wc\" (UID: \"19f5b515-a5a6-42e0-84fb-f77520389f2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-f22wc"
Jul 17 10:07:45.636656 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.636561    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19f5b515-a5a6-42e0-84fb-f77520389f2b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-f22wc\" (UID: \"19f5b515-a5a6-42e0-84fb-f77520389f2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-f22wc"
Jul 17 10:07:45.637140 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.637121    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19f5b515-a5a6-42e0-84fb-f77520389f2b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-f22wc\" (UID: \"19f5b515-a5a6-42e0-84fb-f77520389f2b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-f22wc"
Jul 17 10:07:45.780375 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.780340    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-f22wc"
Jul 17 10:07:45.926723 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:45.926692    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-f22wc"]
Jul 17 10:07:45.927592 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:07:45.927557    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19f5b515_a5a6_42e0_84fb_f77520389f2b.slice/crio-25a06ccccdc6c53404fed3c3099876e0fc8da5b6cee74c5f14ae806ed879fdeb WatchSource:0}: Error finding container 25a06ccccdc6c53404fed3c3099876e0fc8da5b6cee74c5f14ae806ed879fdeb: Status 404 returned error can't find the container with id 25a06ccccdc6c53404fed3c3099876e0fc8da5b6cee74c5f14ae806ed879fdeb
Jul 17 10:07:46.352716 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:46.352677    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-f22wc" event={"ID":"19f5b515-a5a6-42e0-84fb-f77520389f2b","Type":"ContainerStarted","Data":"d4362362b7140d1ae46dce10b30205754b2e675949b923454bc567f7c30a9e3d"}
Jul 17 10:07:46.353312 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:46.352755    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-f22wc" event={"ID":"19f5b515-a5a6-42e0-84fb-f77520389f2b","Type":"ContainerStarted","Data":"25a06ccccdc6c53404fed3c3099876e0fc8da5b6cee74c5f14ae806ed879fdeb"}
Jul 17 10:07:46.376309 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:46.376266    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-f22wc" podStartSLOduration=1.376235258 podStartE2EDuration="1.376235258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:07:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:07:46.373373328 +0000 UTC m=+2463.956011705" watchObservedRunningTime="2026-07-17 10:07:46.376235258 +0000 UTC m=+2463.958873675"
Jul 17 10:07:46.899132 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:46.899041    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8e825ec-7600-4d37-b262-ecba83d00136" path="/var/lib/kubelet/pods/d8e825ec-7600-4d37-b262-ecba83d00136/volumes"
Jul 17 10:07:47.359211 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:47.359177    2574 generic.go:356] "Generic (PLEG): container finished" podID="19f5b515-a5a6-42e0-84fb-f77520389f2b" containerID="d4362362b7140d1ae46dce10b30205754b2e675949b923454bc567f7c30a9e3d" exitCode=6
Jul 17 10:07:47.359627 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:47.359275    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-f22wc" event={"ID":"19f5b515-a5a6-42e0-84fb-f77520389f2b","Type":"ContainerDied","Data":"d4362362b7140d1ae46dce10b30205754b2e675949b923454bc567f7c30a9e3d"}
Jul 17 10:07:47.359627 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:47.359456    2574 scope.go:117] "RemoveContainer" containerID="d4362362b7140d1ae46dce10b30205754b2e675949b923454bc567f7c30a9e3d"
Jul 17 10:07:48.366804 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:48.366766    2574 generic.go:356] "Generic (PLEG): container finished" podID="19f5b515-a5a6-42e0-84fb-f77520389f2b" containerID="3749a49e34d54e84389aabcb508f15aab7b8ed8ccb4da149470d556a7e8dc5a4" exitCode=6
Jul 17 10:07:48.367314 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:48.366856    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-f22wc" event={"ID":"19f5b515-a5a6-42e0-84fb-f77520389f2b","Type":"ContainerDied","Data":"3749a49e34d54e84389aabcb508f15aab7b8ed8ccb4da149470d556a7e8dc5a4"}
Jul 17 10:07:48.367314 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:48.366926    2574 scope.go:117] "RemoveContainer" containerID="d4362362b7140d1ae46dce10b30205754b2e675949b923454bc567f7c30a9e3d"
Jul 17 10:07:48.367314 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:48.367090    2574 scope.go:117] "RemoveContainer" containerID="3749a49e34d54e84389aabcb508f15aab7b8ed8ccb4da149470d556a7e8dc5a4"
Jul 17 10:07:48.367524 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:07:48.367352    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-f22wc_odh-ai-gateway-infra(19f5b515-a5a6-42e0-84fb-f77520389f2b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-f22wc" podUID="19f5b515-a5a6-42e0-84fb-f77520389f2b"
Jul 17 10:07:49.373786 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:49.373762    2574 scope.go:117] "RemoveContainer" containerID="3749a49e34d54e84389aabcb508f15aab7b8ed8ccb4da149470d556a7e8dc5a4"
Jul 17 10:07:49.374272 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:07:49.374054    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-f22wc_odh-ai-gateway-infra(19f5b515-a5a6-42e0-84fb-f77520389f2b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-f22wc" podUID="19f5b515-a5a6-42e0-84fb-f77520389f2b"
Jul 17 10:07:58.214976 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:58.214927    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-vj799"]
Jul 17 10:07:58.215721 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:58.215700    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="49e09a7a-1ebf-4d60-8408-2aab7c42eea5" containerName="revoke-keys"
Jul 17 10:07:58.218871 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:58.218851    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-vj799"
Jul 17 10:07:58.239615 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:58.239591    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-vj799"]
Jul 17 10:07:58.269435 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:58.269407    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffafb729-cea5-4ba4-988a-a47b2b16f9fc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-vj799\" (UID: \"ffafb729-cea5-4ba4-988a-a47b2b16f9fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-vj799"
Jul 17 10:07:58.370118 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:58.370082    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffafb729-cea5-4ba4-988a-a47b2b16f9fc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-vj799\" (UID: \"ffafb729-cea5-4ba4-988a-a47b2b16f9fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-vj799"
Jul 17 10:07:58.370690 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:58.370670    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffafb729-cea5-4ba4-988a-a47b2b16f9fc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-vj799\" (UID: \"ffafb729-cea5-4ba4-988a-a47b2b16f9fc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-vj799"
Jul 17 10:07:58.529426 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:58.529331    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-vj799"
Jul 17 10:07:58.887934 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:58.887905    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-vj799"]
Jul 17 10:07:58.889731 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:07:58.889705    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffafb729_cea5_4ba4_988a_a47b2b16f9fc.slice/crio-bd439b46b8a9cb865f06fe85bd3bfd8de3fd1a84c56f79b7d49dc9507890bf85 WatchSource:0}: Error finding container bd439b46b8a9cb865f06fe85bd3bfd8de3fd1a84c56f79b7d49dc9507890bf85: Status 404 returned error can't find the container with id bd439b46b8a9cb865f06fe85bd3bfd8de3fd1a84c56f79b7d49dc9507890bf85
Jul 17 10:07:59.437843 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:59.437806    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-vj799" event={"ID":"ffafb729-cea5-4ba4-988a-a47b2b16f9fc","Type":"ContainerStarted","Data":"71bcfc9b218264ab92f01a68381c116efa315f8603ca63f7f97028482ca3266f"}
Jul 17 10:07:59.438343 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:59.437870    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-vj799" event={"ID":"ffafb729-cea5-4ba4-988a-a47b2b16f9fc","Type":"ContainerStarted","Data":"bd439b46b8a9cb865f06fe85bd3bfd8de3fd1a84c56f79b7d49dc9507890bf85"}
Jul 17 10:07:59.464837 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:07:59.464748    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-vj799" podStartSLOduration=1.464734273 podStartE2EDuration="1.464734273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:07:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:07:59.46297966 +0000 UTC m=+2477.045618058" watchObservedRunningTime="2026-07-17 10:07:59.464734273 +0000 UTC m=+2477.047372649"
Jul 17 10:08:00.444516 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:00.444474    2574 generic.go:356] "Generic (PLEG): container finished" podID="ffafb729-cea5-4ba4-988a-a47b2b16f9fc" containerID="71bcfc9b218264ab92f01a68381c116efa315f8603ca63f7f97028482ca3266f" exitCode=6
Jul 17 10:08:00.444911 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:00.444562    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-vj799" event={"ID":"ffafb729-cea5-4ba4-988a-a47b2b16f9fc","Type":"ContainerDied","Data":"71bcfc9b218264ab92f01a68381c116efa315f8603ca63f7f97028482ca3266f"}
Jul 17 10:08:00.444911 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:00.444781    2574 scope.go:117] "RemoveContainer" containerID="71bcfc9b218264ab92f01a68381c116efa315f8603ca63f7f97028482ca3266f"
Jul 17 10:08:00.891803 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:00.891765    2574 scope.go:117] "RemoveContainer" containerID="3749a49e34d54e84389aabcb508f15aab7b8ed8ccb4da149470d556a7e8dc5a4"
Jul 17 10:08:01.451995 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:01.451960    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-f22wc" event={"ID":"19f5b515-a5a6-42e0-84fb-f77520389f2b","Type":"ContainerStarted","Data":"48cc7db23f44965e11e5cefa6d63f7dcd2d63480c5bd9f5e20523e8838fc3a6b"}
Jul 17 10:08:01.453956 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:01.453931    2574 generic.go:356] "Generic (PLEG): container finished" podID="ffafb729-cea5-4ba4-988a-a47b2b16f9fc" containerID="03e53cd1c3b0ebe46004fdc5e7149cc250031eeb9a22d6bfc64e0f554e96bd65" exitCode=6
Jul 17 10:08:01.454088 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:01.454014    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-vj799" event={"ID":"ffafb729-cea5-4ba4-988a-a47b2b16f9fc","Type":"ContainerDied","Data":"03e53cd1c3b0ebe46004fdc5e7149cc250031eeb9a22d6bfc64e0f554e96bd65"}
Jul 17 10:08:01.454088 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:01.454078    2574 scope.go:117] "RemoveContainer" containerID="71bcfc9b218264ab92f01a68381c116efa315f8603ca63f7f97028482ca3266f"
Jul 17 10:08:01.454188 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:01.454148    2574 scope.go:117] "RemoveContainer" containerID="03e53cd1c3b0ebe46004fdc5e7149cc250031eeb9a22d6bfc64e0f554e96bd65"
Jul 17 10:08:01.454392 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:08:01.454372    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-vj799_odh-ai-gateway-infra(ffafb729-cea5-4ba4-988a-a47b2b16f9fc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-vj799" podUID="ffafb729-cea5-4ba4-988a-a47b2b16f9fc"
Jul 17 10:08:02.461189 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:02.461158    2574 scope.go:117] "RemoveContainer" containerID="03e53cd1c3b0ebe46004fdc5e7149cc250031eeb9a22d6bfc64e0f554e96bd65"
Jul 17 10:08:02.461656 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:08:02.461445    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-vj799_odh-ai-gateway-infra(ffafb729-cea5-4ba4-988a-a47b2b16f9fc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-vj799" podUID="ffafb729-cea5-4ba4-988a-a47b2b16f9fc"
Jul 17 10:08:02.462918 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:02.462892    2574 generic.go:356] "Generic (PLEG): container finished" podID="19f5b515-a5a6-42e0-84fb-f77520389f2b" containerID="48cc7db23f44965e11e5cefa6d63f7dcd2d63480c5bd9f5e20523e8838fc3a6b" exitCode=6
Jul 17 10:08:02.462918 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:02.462927    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-f22wc" event={"ID":"19f5b515-a5a6-42e0-84fb-f77520389f2b","Type":"ContainerDied","Data":"48cc7db23f44965e11e5cefa6d63f7dcd2d63480c5bd9f5e20523e8838fc3a6b"}
Jul 17 10:08:02.463126 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:02.463012    2574 scope.go:117] "RemoveContainer" containerID="3749a49e34d54e84389aabcb508f15aab7b8ed8ccb4da149470d556a7e8dc5a4"
Jul 17 10:08:02.463384 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:02.463368    2574 scope.go:117] "RemoveContainer" containerID="48cc7db23f44965e11e5cefa6d63f7dcd2d63480c5bd9f5e20523e8838fc3a6b"
Jul 17 10:08:02.463579 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:08:02.463559    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-f22wc_odh-ai-gateway-infra(19f5b515-a5a6-42e0-84fb-f77520389f2b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-f22wc" podUID="19f5b515-a5a6-42e0-84fb-f77520389f2b"
Jul 17 10:08:02.487674 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:02.487640    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-f22wc"]
Jul 17 10:08:03.623234 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:03.623205    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-f22wc"
Jul 17 10:08:03.721726 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:03.721629    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19f5b515-a5a6-42e0-84fb-f77520389f2b-maas-api-service-ca\") pod \"19f5b515-a5a6-42e0-84fb-f77520389f2b\" (UID: \"19f5b515-a5a6-42e0-84fb-f77520389f2b\") "
Jul 17 10:08:03.722051 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:03.722025    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/19f5b515-a5a6-42e0-84fb-f77520389f2b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "19f5b515-a5a6-42e0-84fb-f77520389f2b" (UID: "19f5b515-a5a6-42e0-84fb-f77520389f2b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:08:03.822357 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:03.822324    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/19f5b515-a5a6-42e0-84fb-f77520389f2b-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:08:04.476125 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:04.476089    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-f22wc" event={"ID":"19f5b515-a5a6-42e0-84fb-f77520389f2b","Type":"ContainerDied","Data":"25a06ccccdc6c53404fed3c3099876e0fc8da5b6cee74c5f14ae806ed879fdeb"}
Jul 17 10:08:04.476125 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:04.476116    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-f22wc"
Jul 17 10:08:04.476369 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:04.476156    2574 scope.go:117] "RemoveContainer" containerID="48cc7db23f44965e11e5cefa6d63f7dcd2d63480c5bd9f5e20523e8838fc3a6b"
Jul 17 10:08:04.498740 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:04.498717    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-f22wc"]
Jul 17 10:08:04.520696 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:04.520668    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-f22wc"]
Jul 17 10:08:04.898136 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:04.898053    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19f5b515-a5a6-42e0-84fb-f77520389f2b" path="/var/lib/kubelet/pods/19f5b515-a5a6-42e0-84fb-f77520389f2b/volumes"
Jul 17 10:08:16.088592 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:16.088552    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-97kd2"]
Jul 17 10:08:16.089029 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:16.089016    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19f5b515-a5a6-42e0-84fb-f77520389f2b" containerName="revoke-keys"
Jul 17 10:08:16.089029 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:16.089030    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="19f5b515-a5a6-42e0-84fb-f77520389f2b" containerName="revoke-keys"
Jul 17 10:08:16.089104 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:16.089056    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19f5b515-a5a6-42e0-84fb-f77520389f2b" containerName="revoke-keys"
Jul 17 10:08:16.089104 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:16.089063    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="19f5b515-a5a6-42e0-84fb-f77520389f2b" containerName="revoke-keys"
Jul 17 10:08:16.089166 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:16.089129    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="19f5b515-a5a6-42e0-84fb-f77520389f2b" containerName="revoke-keys"
Jul 17 10:08:16.089166 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:16.089137    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="19f5b515-a5a6-42e0-84fb-f77520389f2b" containerName="revoke-keys"
Jul 17 10:08:16.089166 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:16.089144    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="19f5b515-a5a6-42e0-84fb-f77520389f2b" containerName="revoke-keys"
Jul 17 10:08:16.094052 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:16.094028    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-97kd2"
Jul 17 10:08:16.100937 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:16.100896    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-97kd2"]
Jul 17 10:08:16.137264 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:16.137208    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fe01889-9a26-40c7-8a9e-4213f1cba8a1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-97kd2\" (UID: \"5fe01889-9a26-40c7-8a9e-4213f1cba8a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-97kd2"
Jul 17 10:08:16.237976 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:16.237936    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fe01889-9a26-40c7-8a9e-4213f1cba8a1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-97kd2\" (UID: \"5fe01889-9a26-40c7-8a9e-4213f1cba8a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-97kd2"
Jul 17 10:08:16.238670 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:16.238642    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fe01889-9a26-40c7-8a9e-4213f1cba8a1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-97kd2\" (UID: \"5fe01889-9a26-40c7-8a9e-4213f1cba8a1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-97kd2"
Jul 17 10:08:16.407268 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:16.407171    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-97kd2"
Jul 17 10:08:16.554976 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:16.554945    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-97kd2"]
Jul 17 10:08:16.558082 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:08:16.558057    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fe01889_9a26_40c7_8a9e_4213f1cba8a1.slice/crio-bf80f37bb096c134eae57541e197cdfc64e87eff213fe009081656909ccd1a4c WatchSource:0}: Error finding container bf80f37bb096c134eae57541e197cdfc64e87eff213fe009081656909ccd1a4c: Status 404 returned error can't find the container with id bf80f37bb096c134eae57541e197cdfc64e87eff213fe009081656909ccd1a4c
Jul 17 10:08:17.144183 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:17.144081    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mnfph"]
Jul 17 10:08:17.144838 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:17.144813    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19f5b515-a5a6-42e0-84fb-f77520389f2b" containerName="revoke-keys"
Jul 17 10:08:17.144935 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:17.144841    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="19f5b515-a5a6-42e0-84fb-f77520389f2b" containerName="revoke-keys"
Jul 17 10:08:17.148217 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:17.148195    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mnfph"
Jul 17 10:08:17.156721 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:17.156693    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mnfph"]
Jul 17 10:08:17.249019 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:17.248987    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ab65f52-472b-4963-be7c-f721e3c51153-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-mnfph\" (UID: \"9ab65f52-472b-4963-be7c-f721e3c51153\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mnfph"
Jul 17 10:08:17.350082 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:17.350055    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ab65f52-472b-4963-be7c-f721e3c51153-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-mnfph\" (UID: \"9ab65f52-472b-4963-be7c-f721e3c51153\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mnfph"
Jul 17 10:08:17.350699 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:17.350680    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ab65f52-472b-4963-be7c-f721e3c51153-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-mnfph\" (UID: \"9ab65f52-472b-4963-be7c-f721e3c51153\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mnfph"
Jul 17 10:08:17.460328 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:17.460298    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mnfph"
Jul 17 10:08:17.556166 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:17.556136    2574 generic.go:356] "Generic (PLEG): container finished" podID="5fe01889-9a26-40c7-8a9e-4213f1cba8a1" containerID="f7b7dbff6c4ed2b018160a77374d39c44586e3a0b0592dc425e225234302be03" exitCode=6
Jul 17 10:08:17.556366 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:17.556298    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-97kd2" event={"ID":"5fe01889-9a26-40c7-8a9e-4213f1cba8a1","Type":"ContainerDied","Data":"f7b7dbff6c4ed2b018160a77374d39c44586e3a0b0592dc425e225234302be03"}
Jul 17 10:08:17.556366 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:17.556345    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-97kd2" event={"ID":"5fe01889-9a26-40c7-8a9e-4213f1cba8a1","Type":"ContainerStarted","Data":"bf80f37bb096c134eae57541e197cdfc64e87eff213fe009081656909ccd1a4c"}
Jul 17 10:08:17.556611 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:17.556591    2574 scope.go:117] "RemoveContainer" containerID="f7b7dbff6c4ed2b018160a77374d39c44586e3a0b0592dc425e225234302be03"
Jul 17 10:08:17.614097 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:17.614075    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mnfph"]
Jul 17 10:08:17.616648 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:08:17.616621    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ab65f52_472b_4963_be7c_f721e3c51153.slice/crio-db7b9996963a71ec73d9d1516627e7e03b3f7a641d6333e773e0d9322f634c5a WatchSource:0}: Error finding container db7b9996963a71ec73d9d1516627e7e03b3f7a641d6333e773e0d9322f634c5a: Status 404 returned error can't find the container with id db7b9996963a71ec73d9d1516627e7e03b3f7a641d6333e773e0d9322f634c5a
Jul 17 10:08:17.891410 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:17.891372    2574 scope.go:117] "RemoveContainer" containerID="03e53cd1c3b0ebe46004fdc5e7149cc250031eeb9a22d6bfc64e0f554e96bd65"
Jul 17 10:08:18.563881 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:18.563847    2574 generic.go:356] "Generic (PLEG): container finished" podID="9ab65f52-472b-4963-be7c-f721e3c51153" containerID="c4ea5f2d65b40f0aac04897be842f8b543750b7fb0a8d48d47da80ddf7e2fc32" exitCode=6
Jul 17 10:08:18.564328 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:18.563931    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mnfph" event={"ID":"9ab65f52-472b-4963-be7c-f721e3c51153","Type":"ContainerDied","Data":"c4ea5f2d65b40f0aac04897be842f8b543750b7fb0a8d48d47da80ddf7e2fc32"}
Jul 17 10:08:18.564328 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:18.563980    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mnfph" event={"ID":"9ab65f52-472b-4963-be7c-f721e3c51153","Type":"ContainerStarted","Data":"db7b9996963a71ec73d9d1516627e7e03b3f7a641d6333e773e0d9322f634c5a"}
Jul 17 10:08:18.564328 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:18.564105    2574 scope.go:117] "RemoveContainer" containerID="c4ea5f2d65b40f0aac04897be842f8b543750b7fb0a8d48d47da80ddf7e2fc32"
Jul 17 10:08:18.565965 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:18.565944    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-vj799" event={"ID":"ffafb729-cea5-4ba4-988a-a47b2b16f9fc","Type":"ContainerStarted","Data":"79ff8562a1a47ebca6898db9055ba2b4b1c641c4e36a4515b1277194343e6cfd"}
Jul 17 10:08:18.567858 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:18.567840    2574 generic.go:356] "Generic (PLEG): container finished" podID="5fe01889-9a26-40c7-8a9e-4213f1cba8a1" containerID="e1d0dfcb61f492531b1baa113a3df5b1b0f89e4eab340da2daa936d01558a722" exitCode=6
Jul 17 10:08:18.567938 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:18.567905    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-97kd2" event={"ID":"5fe01889-9a26-40c7-8a9e-4213f1cba8a1","Type":"ContainerDied","Data":"e1d0dfcb61f492531b1baa113a3df5b1b0f89e4eab340da2daa936d01558a722"}
Jul 17 10:08:18.568062 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:18.567966    2574 scope.go:117] "RemoveContainer" containerID="f7b7dbff6c4ed2b018160a77374d39c44586e3a0b0592dc425e225234302be03"
Jul 17 10:08:18.568062 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:18.568039    2574 scope.go:117] "RemoveContainer" containerID="e1d0dfcb61f492531b1baa113a3df5b1b0f89e4eab340da2daa936d01558a722"
Jul 17 10:08:18.568266 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:08:18.568221    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-97kd2_odh-ai-gateway-infra(5fe01889-9a26-40c7-8a9e-4213f1cba8a1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-97kd2" podUID="5fe01889-9a26-40c7-8a9e-4213f1cba8a1"
Jul 17 10:08:19.575641 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:19.575536    2574 generic.go:356] "Generic (PLEG): container finished" podID="9ab65f52-472b-4963-be7c-f721e3c51153" containerID="f9c3c7aaed278b0e99345f35d339aaec0aa01a33a5942879e1468e8a905e8b6e" exitCode=6
Jul 17 10:08:19.576064 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:19.575627    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mnfph" event={"ID":"9ab65f52-472b-4963-be7c-f721e3c51153","Type":"ContainerDied","Data":"f9c3c7aaed278b0e99345f35d339aaec0aa01a33a5942879e1468e8a905e8b6e"}
Jul 17 10:08:19.576064 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:19.575705    2574 scope.go:117] "RemoveContainer" containerID="c4ea5f2d65b40f0aac04897be842f8b543750b7fb0a8d48d47da80ddf7e2fc32"
Jul 17 10:08:19.576064 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:19.575865    2574 scope.go:117] "RemoveContainer" containerID="f9c3c7aaed278b0e99345f35d339aaec0aa01a33a5942879e1468e8a905e8b6e"
Jul 17 10:08:19.576215 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:08:19.576127    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-mnfph_odh-ai-gateway-infra(9ab65f52-472b-4963-be7c-f721e3c51153)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mnfph" podUID="9ab65f52-472b-4963-be7c-f721e3c51153"
Jul 17 10:08:19.577758 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:19.577738    2574 generic.go:356] "Generic (PLEG): container finished" podID="ffafb729-cea5-4ba4-988a-a47b2b16f9fc" containerID="79ff8562a1a47ebca6898db9055ba2b4b1c641c4e36a4515b1277194343e6cfd" exitCode=6
Jul 17 10:08:19.577869 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:19.577813    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-vj799" event={"ID":"ffafb729-cea5-4ba4-988a-a47b2b16f9fc","Type":"ContainerDied","Data":"79ff8562a1a47ebca6898db9055ba2b4b1c641c4e36a4515b1277194343e6cfd"}
Jul 17 10:08:19.578130 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:19.578115    2574 scope.go:117] "RemoveContainer" containerID="79ff8562a1a47ebca6898db9055ba2b4b1c641c4e36a4515b1277194343e6cfd"
Jul 17 10:08:19.578357 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:08:19.578335    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-vj799_odh-ai-gateway-infra(ffafb729-cea5-4ba4-988a-a47b2b16f9fc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-vj799" podUID="ffafb729-cea5-4ba4-988a-a47b2b16f9fc"
Jul 17 10:08:19.579627 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:19.579610    2574 scope.go:117] "RemoveContainer" containerID="e1d0dfcb61f492531b1baa113a3df5b1b0f89e4eab340da2daa936d01558a722"
Jul 17 10:08:19.579808 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:08:19.579786    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-97kd2_odh-ai-gateway-infra(5fe01889-9a26-40c7-8a9e-4213f1cba8a1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-97kd2" podUID="5fe01889-9a26-40c7-8a9e-4213f1cba8a1"
Jul 17 10:08:19.587233 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:19.587218    2574 scope.go:117] "RemoveContainer" containerID="03e53cd1c3b0ebe46004fdc5e7149cc250031eeb9a22d6bfc64e0f554e96bd65"
Jul 17 10:08:19.649965 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:19.649934    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-vj799"]
Jul 17 10:08:20.587100 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:20.587070    2574 scope.go:117] "RemoveContainer" containerID="f9c3c7aaed278b0e99345f35d339aaec0aa01a33a5942879e1468e8a905e8b6e"
Jul 17 10:08:20.587579 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:08:20.587332    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-mnfph_odh-ai-gateway-infra(9ab65f52-472b-4963-be7c-f721e3c51153)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mnfph" podUID="9ab65f52-472b-4963-be7c-f721e3c51153"
Jul 17 10:08:20.732859 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:20.732831    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-vj799"
Jul 17 10:08:20.783936 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:20.783907    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffafb729-cea5-4ba4-988a-a47b2b16f9fc-maas-api-service-ca\") pod \"ffafb729-cea5-4ba4-988a-a47b2b16f9fc\" (UID: \"ffafb729-cea5-4ba4-988a-a47b2b16f9fc\") "
Jul 17 10:08:20.784360 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:20.784328    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffafb729-cea5-4ba4-988a-a47b2b16f9fc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ffafb729-cea5-4ba4-988a-a47b2b16f9fc" (UID: "ffafb729-cea5-4ba4-988a-a47b2b16f9fc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:08:20.884762 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:20.884681    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ffafb729-cea5-4ba4-988a-a47b2b16f9fc-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:08:21.595280 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:21.595227    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-vj799"
Jul 17 10:08:21.595280 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:21.595232    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-vj799" event={"ID":"ffafb729-cea5-4ba4-988a-a47b2b16f9fc","Type":"ContainerDied","Data":"bd439b46b8a9cb865f06fe85bd3bfd8de3fd1a84c56f79b7d49dc9507890bf85"}
Jul 17 10:08:21.595801 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:21.595324    2574 scope.go:117] "RemoveContainer" containerID="79ff8562a1a47ebca6898db9055ba2b4b1c641c4e36a4515b1277194343e6cfd"
Jul 17 10:08:21.614678 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:21.614652    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-vj799"]
Jul 17 10:08:21.621805 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:21.621772    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-vj799"]
Jul 17 10:08:22.897682 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:22.897652    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ffafb729-cea5-4ba4-988a-a47b2b16f9fc" path="/var/lib/kubelet/pods/ffafb729-cea5-4ba4-988a-a47b2b16f9fc/volumes"
Jul 17 10:08:31.985035 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:31.984997    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cda038a0-fcqbz"]
Jul 17 10:08:31.985559 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:31.985540    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffafb729-cea5-4ba4-988a-a47b2b16f9fc" containerName="revoke-keys"
Jul 17 10:08:31.985559 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:31.985554    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffafb729-cea5-4ba4-988a-a47b2b16f9fc" containerName="revoke-keys"
Jul 17 10:08:31.985634 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:31.985564    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffafb729-cea5-4ba4-988a-a47b2b16f9fc" containerName="revoke-keys"
Jul 17 10:08:31.985634 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:31.985570    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffafb729-cea5-4ba4-988a-a47b2b16f9fc" containerName="revoke-keys"
Jul 17 10:08:31.985704 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:31.985645    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffafb729-cea5-4ba4-988a-a47b2b16f9fc" containerName="revoke-keys"
Jul 17 10:08:31.985704 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:31.985652    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffafb729-cea5-4ba4-988a-a47b2b16f9fc" containerName="revoke-keys"
Jul 17 10:08:31.985704 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:31.985659    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffafb729-cea5-4ba4-988a-a47b2b16f9fc" containerName="revoke-keys"
Jul 17 10:08:31.990507 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:31.990489    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cda038a0-fcqbz"
Jul 17 10:08:32.005448 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:32.005419    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cda038a0-fcqbz"]
Jul 17 10:08:32.098435 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:32.098398    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f61c2442-1e0c-4943-b032-e8ff24d3eeca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-cda038a0-fcqbz\" (UID: \"f61c2442-1e0c-4943-b032-e8ff24d3eeca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cda038a0-fcqbz"
Jul 17 10:08:32.199633 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:32.199602    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f61c2442-1e0c-4943-b032-e8ff24d3eeca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-cda038a0-fcqbz\" (UID: \"f61c2442-1e0c-4943-b032-e8ff24d3eeca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cda038a0-fcqbz"
Jul 17 10:08:32.200205 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:32.200182    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f61c2442-1e0c-4943-b032-e8ff24d3eeca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-cda038a0-fcqbz\" (UID: \"f61c2442-1e0c-4943-b032-e8ff24d3eeca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cda038a0-fcqbz"
Jul 17 10:08:32.301147 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:32.301055    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cda038a0-fcqbz"
Jul 17 10:08:32.449176 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:32.449149    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cda038a0-fcqbz"]
Jul 17 10:08:32.451191 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:08:32.451160    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf61c2442_1e0c_4943_b032_e8ff24d3eeca.slice/crio-96b522192d26aa16c63aa4954448720400ed383ed41be30270b4f7adc1d0902a WatchSource:0}: Error finding container 96b522192d26aa16c63aa4954448720400ed383ed41be30270b4f7adc1d0902a: Status 404 returned error can't find the container with id 96b522192d26aa16c63aa4954448720400ed383ed41be30270b4f7adc1d0902a
Jul 17 10:08:32.659860 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:32.659826    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cda038a0-fcqbz" event={"ID":"f61c2442-1e0c-4943-b032-e8ff24d3eeca","Type":"ContainerStarted","Data":"96b522192d26aa16c63aa4954448720400ed383ed41be30270b4f7adc1d0902a"}
Jul 17 10:08:33.666955 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:33.666924    2574 generic.go:356] "Generic (PLEG): container finished" podID="f61c2442-1e0c-4943-b032-e8ff24d3eeca" containerID="65afb001ea31c45fa6b53f7261e57304d5ec4ffeef0911a9eecbb5fcac05a70a" exitCode=0
Jul 17 10:08:33.667339 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:33.666974    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cda038a0-fcqbz" event={"ID":"f61c2442-1e0c-4943-b032-e8ff24d3eeca","Type":"ContainerDied","Data":"65afb001ea31c45fa6b53f7261e57304d5ec4ffeef0911a9eecbb5fcac05a70a"}
Jul 17 10:08:33.891337 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:33.891308    2574 scope.go:117] "RemoveContainer" containerID="e1d0dfcb61f492531b1baa113a3df5b1b0f89e4eab340da2daa936d01558a722"
Jul 17 10:08:34.675559 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:34.675526    2574 generic.go:356] "Generic (PLEG): container finished" podID="5fe01889-9a26-40c7-8a9e-4213f1cba8a1" containerID="6ed1b685983f5e3adb0a9166212202b805f57d27810f7972b3effdb5b0ba704b" exitCode=6
Jul 17 10:08:34.675994 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:34.675604    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-97kd2" event={"ID":"5fe01889-9a26-40c7-8a9e-4213f1cba8a1","Type":"ContainerDied","Data":"6ed1b685983f5e3adb0a9166212202b805f57d27810f7972b3effdb5b0ba704b"}
Jul 17 10:08:34.675994 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:34.675681    2574 scope.go:117] "RemoveContainer" containerID="e1d0dfcb61f492531b1baa113a3df5b1b0f89e4eab340da2daa936d01558a722"
Jul 17 10:08:34.676174 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:34.676046    2574 scope.go:117] "RemoveContainer" containerID="6ed1b685983f5e3adb0a9166212202b805f57d27810f7972b3effdb5b0ba704b"
Jul 17 10:08:34.676396 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:08:34.676374    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-97kd2_odh-ai-gateway-infra(5fe01889-9a26-40c7-8a9e-4213f1cba8a1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-97kd2" podUID="5fe01889-9a26-40c7-8a9e-4213f1cba8a1"
Jul 17 10:08:34.820935 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:34.820906    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cda038a0-fcqbz"
Jul 17 10:08:34.891545 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:34.891457    2574 scope.go:117] "RemoveContainer" containerID="f9c3c7aaed278b0e99345f35d339aaec0aa01a33a5942879e1468e8a905e8b6e"
Jul 17 10:08:34.925627 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:34.925606    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f61c2442-1e0c-4943-b032-e8ff24d3eeca-maas-api-service-ca\") pod \"f61c2442-1e0c-4943-b032-e8ff24d3eeca\" (UID: \"f61c2442-1e0c-4943-b032-e8ff24d3eeca\") "
Jul 17 10:08:34.925934 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:34.925912    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f61c2442-1e0c-4943-b032-e8ff24d3eeca-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f61c2442-1e0c-4943-b032-e8ff24d3eeca" (UID: "f61c2442-1e0c-4943-b032-e8ff24d3eeca"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:08:35.026876 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:35.026846    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f61c2442-1e0c-4943-b032-e8ff24d3eeca-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:08:35.513670 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:35.513640    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fqpqr"]
Jul 17 10:08:35.514145 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:35.514132    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f61c2442-1e0c-4943-b032-e8ff24d3eeca" containerName="revoke-keys"
Jul 17 10:08:35.514197 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:35.514147    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f61c2442-1e0c-4943-b032-e8ff24d3eeca" containerName="revoke-keys"
Jul 17 10:08:35.514197 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:35.514157    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffafb729-cea5-4ba4-988a-a47b2b16f9fc" containerName="revoke-keys"
Jul 17 10:08:35.514197 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:35.514164    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffafb729-cea5-4ba4-988a-a47b2b16f9fc" containerName="revoke-keys"
Jul 17 10:08:35.514400 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:35.514234    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f61c2442-1e0c-4943-b032-e8ff24d3eeca" containerName="revoke-keys"
Jul 17 10:08:35.520354 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:35.520327    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fqpqr"
Jul 17 10:08:35.526805 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:35.526778    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fqpqr"]
Jul 17 10:08:35.633912 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:35.633876    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18a98949-fec9-4b12-a6b0-664275904a47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-fqpqr\" (UID: \"18a98949-fec9-4b12-a6b0-664275904a47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fqpqr"
Jul 17 10:08:35.685228 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:35.685198    2574 generic.go:356] "Generic (PLEG): container finished" podID="9ab65f52-472b-4963-be7c-f721e3c51153" containerID="f9f8435e281a3969841b141f05531d1a1bfbe744f6016a64145e94faa423db70" exitCode=6
Jul 17 10:08:35.685651 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:35.685274    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mnfph" event={"ID":"9ab65f52-472b-4963-be7c-f721e3c51153","Type":"ContainerDied","Data":"f9f8435e281a3969841b141f05531d1a1bfbe744f6016a64145e94faa423db70"}
Jul 17 10:08:35.685651 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:35.685337    2574 scope.go:117] "RemoveContainer" containerID="f9c3c7aaed278b0e99345f35d339aaec0aa01a33a5942879e1468e8a905e8b6e"
Jul 17 10:08:35.685816 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:35.685714    2574 scope.go:117] "RemoveContainer" containerID="f9f8435e281a3969841b141f05531d1a1bfbe744f6016a64145e94faa423db70"
Jul 17 10:08:35.686001 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:08:35.685977    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-mnfph_odh-ai-gateway-infra(9ab65f52-472b-4963-be7c-f721e3c51153)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mnfph" podUID="9ab65f52-472b-4963-be7c-f721e3c51153"
Jul 17 10:08:35.686712 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:35.686689    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cda038a0-fcqbz" event={"ID":"f61c2442-1e0c-4943-b032-e8ff24d3eeca","Type":"ContainerDied","Data":"96b522192d26aa16c63aa4954448720400ed383ed41be30270b4f7adc1d0902a"}
Jul 17 10:08:35.686796 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:35.686738    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="96b522192d26aa16c63aa4954448720400ed383ed41be30270b4f7adc1d0902a"
Jul 17 10:08:35.686796 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:35.686703    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-cda038a0-fcqbz"
Jul 17 10:08:35.703938 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:35.703917    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-97kd2"]
Jul 17 10:08:35.734953 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:35.734920    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18a98949-fec9-4b12-a6b0-664275904a47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-fqpqr\" (UID: \"18a98949-fec9-4b12-a6b0-664275904a47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fqpqr"
Jul 17 10:08:35.735635 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:35.735607    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18a98949-fec9-4b12-a6b0-664275904a47-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-fqpqr\" (UID: \"18a98949-fec9-4b12-a6b0-664275904a47\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fqpqr"
Jul 17 10:08:35.847273 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:35.847233    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-97kd2"
Jul 17 10:08:35.897476 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:35.897442    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fqpqr"
Jul 17 10:08:36.045439 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:36.045335    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fe01889-9a26-40c7-8a9e-4213f1cba8a1-maas-api-service-ca\") pod \"5fe01889-9a26-40c7-8a9e-4213f1cba8a1\" (UID: \"5fe01889-9a26-40c7-8a9e-4213f1cba8a1\") "
Jul 17 10:08:36.046313 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:36.046281    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fe01889-9a26-40c7-8a9e-4213f1cba8a1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5fe01889-9a26-40c7-8a9e-4213f1cba8a1" (UID: "5fe01889-9a26-40c7-8a9e-4213f1cba8a1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:08:36.147157 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:36.147120    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fe01889-9a26-40c7-8a9e-4213f1cba8a1-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:08:36.252831 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:36.252805    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fqpqr"]
Jul 17 10:08:36.254982 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:08:36.254954    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18a98949_fec9_4b12_a6b0_664275904a47.slice/crio-e9a9a78d20553838eb1deafba573aecc27c3ff98fee423e2a3ab44b9dd232717 WatchSource:0}: Error finding container e9a9a78d20553838eb1deafba573aecc27c3ff98fee423e2a3ab44b9dd232717: Status 404 returned error can't find the container with id e9a9a78d20553838eb1deafba573aecc27c3ff98fee423e2a3ab44b9dd232717
Jul 17 10:08:36.695307 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:36.695253    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fqpqr" event={"ID":"18a98949-fec9-4b12-a6b0-664275904a47","Type":"ContainerStarted","Data":"2aa38ae97b97c33723f0f6bb773ea9d7d62314b3ae92d6db74864e50c9e8067f"}
Jul 17 10:08:36.695782 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:36.695349    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fqpqr" event={"ID":"18a98949-fec9-4b12-a6b0-664275904a47","Type":"ContainerStarted","Data":"e9a9a78d20553838eb1deafba573aecc27c3ff98fee423e2a3ab44b9dd232717"}
Jul 17 10:08:36.696609 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:36.696587    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-97kd2" event={"ID":"5fe01889-9a26-40c7-8a9e-4213f1cba8a1","Type":"ContainerDied","Data":"bf80f37bb096c134eae57541e197cdfc64e87eff213fe009081656909ccd1a4c"}
Jul 17 10:08:36.696609 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:36.696605    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-97kd2"
Jul 17 10:08:36.696738 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:36.696636    2574 scope.go:117] "RemoveContainer" containerID="6ed1b685983f5e3adb0a9166212202b805f57d27810f7972b3effdb5b0ba704b"
Jul 17 10:08:36.715301 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:36.715261    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fqpqr" podStartSLOduration=1.7152257419999999 podStartE2EDuration="1.715225742s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:08:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:08:36.713941082 +0000 UTC m=+2514.296579458" watchObservedRunningTime="2026-07-17 10:08:36.715225742 +0000 UTC m=+2514.297864133"
Jul 17 10:08:36.718923 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:36.718899    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mnfph"]
Jul 17 10:08:36.730287 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:36.730260    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-97kd2"]
Jul 17 10:08:36.739970 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:36.739946    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-97kd2"]
Jul 17 10:08:36.851371 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:36.851350    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mnfph"
Jul 17 10:08:36.854661 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:36.854638    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ab65f52-472b-4963-be7c-f721e3c51153-maas-api-service-ca\") pod \"9ab65f52-472b-4963-be7c-f721e3c51153\" (UID: \"9ab65f52-472b-4963-be7c-f721e3c51153\") "
Jul 17 10:08:36.854999 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:36.854974    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ab65f52-472b-4963-be7c-f721e3c51153-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9ab65f52-472b-4963-be7c-f721e3c51153" (UID: "9ab65f52-472b-4963-be7c-f721e3c51153"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:08:36.899178 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:36.899109    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fe01889-9a26-40c7-8a9e-4213f1cba8a1" path="/var/lib/kubelet/pods/5fe01889-9a26-40c7-8a9e-4213f1cba8a1/volumes"
Jul 17 10:08:36.955913 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:36.955884    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ab65f52-472b-4963-be7c-f721e3c51153-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:08:37.705515 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:37.705485    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mnfph"
Jul 17 10:08:37.705515 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:37.705486    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mnfph" event={"ID":"9ab65f52-472b-4963-be7c-f721e3c51153","Type":"ContainerDied","Data":"db7b9996963a71ec73d9d1516627e7e03b3f7a641d6333e773e0d9322f634c5a"}
Jul 17 10:08:37.705973 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:37.705574    2574 scope.go:117] "RemoveContainer" containerID="f9f8435e281a3969841b141f05531d1a1bfbe744f6016a64145e94faa423db70"
Jul 17 10:08:37.708154 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:37.707957    2574 generic.go:356] "Generic (PLEG): container finished" podID="18a98949-fec9-4b12-a6b0-664275904a47" containerID="2aa38ae97b97c33723f0f6bb773ea9d7d62314b3ae92d6db74864e50c9e8067f" exitCode=6
Jul 17 10:08:37.708154 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:37.708053    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fqpqr" event={"ID":"18a98949-fec9-4b12-a6b0-664275904a47","Type":"ContainerDied","Data":"2aa38ae97b97c33723f0f6bb773ea9d7d62314b3ae92d6db74864e50c9e8067f"}
Jul 17 10:08:37.708502 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:37.708230    2574 scope.go:117] "RemoveContainer" containerID="2aa38ae97b97c33723f0f6bb773ea9d7d62314b3ae92d6db74864e50c9e8067f"
Jul 17 10:08:37.745335 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:37.745307    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mnfph"]
Jul 17 10:08:37.750558 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:37.750538    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-mnfph"]
Jul 17 10:08:38.716085 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:38.716052    2574 generic.go:356] "Generic (PLEG): container finished" podID="18a98949-fec9-4b12-a6b0-664275904a47" containerID="28771c13a5719ec8d9284cc6fc18c7d6b3bf5b1ff721e3322a98c765b68f813b" exitCode=6
Jul 17 10:08:38.716546 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:38.716145    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fqpqr" event={"ID":"18a98949-fec9-4b12-a6b0-664275904a47","Type":"ContainerDied","Data":"28771c13a5719ec8d9284cc6fc18c7d6b3bf5b1ff721e3322a98c765b68f813b"}
Jul 17 10:08:38.716546 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:38.716207    2574 scope.go:117] "RemoveContainer" containerID="2aa38ae97b97c33723f0f6bb773ea9d7d62314b3ae92d6db74864e50c9e8067f"
Jul 17 10:08:38.716546 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:38.716312    2574 scope.go:117] "RemoveContainer" containerID="28771c13a5719ec8d9284cc6fc18c7d6b3bf5b1ff721e3322a98c765b68f813b"
Jul 17 10:08:38.716672 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:08:38.716598    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-fqpqr_odh-ai-gateway-infra(18a98949-fec9-4b12-a6b0-664275904a47)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fqpqr" podUID="18a98949-fec9-4b12-a6b0-664275904a47"
Jul 17 10:08:38.897831 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:38.897787    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ab65f52-472b-4963-be7c-f721e3c51153" path="/var/lib/kubelet/pods/9ab65f52-472b-4963-be7c-f721e3c51153/volumes"
Jul 17 10:08:39.723471 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:39.723447    2574 scope.go:117] "RemoveContainer" containerID="28771c13a5719ec8d9284cc6fc18c7d6b3bf5b1ff721e3322a98c765b68f813b"
Jul 17 10:08:39.723882 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:08:39.723664    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-fqpqr_odh-ai-gateway-infra(18a98949-fec9-4b12-a6b0-664275904a47)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fqpqr" podUID="18a98949-fec9-4b12-a6b0-664275904a47"
Jul 17 10:08:51.891773 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:51.891743    2574 scope.go:117] "RemoveContainer" containerID="28771c13a5719ec8d9284cc6fc18c7d6b3bf5b1ff721e3322a98c765b68f813b"
Jul 17 10:08:52.793273 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:52.793224    2574 generic.go:356] "Generic (PLEG): container finished" podID="18a98949-fec9-4b12-a6b0-664275904a47" containerID="49067a1a30810b4abd1d858a7afa13916d4e116984206e1c26959db43577974c" exitCode=6
Jul 17 10:08:52.793443 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:52.793296    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fqpqr" event={"ID":"18a98949-fec9-4b12-a6b0-664275904a47","Type":"ContainerDied","Data":"49067a1a30810b4abd1d858a7afa13916d4e116984206e1c26959db43577974c"}
Jul 17 10:08:52.793443 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:52.793362    2574 scope.go:117] "RemoveContainer" containerID="28771c13a5719ec8d9284cc6fc18c7d6b3bf5b1ff721e3322a98c765b68f813b"
Jul 17 10:08:52.793715 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:52.793695    2574 scope.go:117] "RemoveContainer" containerID="49067a1a30810b4abd1d858a7afa13916d4e116984206e1c26959db43577974c"
Jul 17 10:08:52.793976 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:08:52.793955    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-fqpqr_odh-ai-gateway-infra(18a98949-fec9-4b12-a6b0-664275904a47)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fqpqr" podUID="18a98949-fec9-4b12-a6b0-664275904a47"
Jul 17 10:08:53.826262 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:53.826211    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fqpqr"]
Jul 17 10:08:53.992092 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:53.992064    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fqpqr"
Jul 17 10:08:54.131075 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:54.130983    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18a98949-fec9-4b12-a6b0-664275904a47-maas-api-service-ca\") pod \"18a98949-fec9-4b12-a6b0-664275904a47\" (UID: \"18a98949-fec9-4b12-a6b0-664275904a47\") "
Jul 17 10:08:54.131435 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:54.131408    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18a98949-fec9-4b12-a6b0-664275904a47-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "18a98949-fec9-4b12-a6b0-664275904a47" (UID: "18a98949-fec9-4b12-a6b0-664275904a47"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:08:54.231977 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:54.231943    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/18a98949-fec9-4b12-a6b0-664275904a47-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:08:54.813676 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:54.813640    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fqpqr"
Jul 17 10:08:54.813676 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:54.813664    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fqpqr" event={"ID":"18a98949-fec9-4b12-a6b0-664275904a47","Type":"ContainerDied","Data":"e9a9a78d20553838eb1deafba573aecc27c3ff98fee423e2a3ab44b9dd232717"}
Jul 17 10:08:54.813913 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:54.813728    2574 scope.go:117] "RemoveContainer" containerID="49067a1a30810b4abd1d858a7afa13916d4e116984206e1c26959db43577974c"
Jul 17 10:08:54.841218 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:54.841191    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fqpqr"]
Jul 17 10:08:54.850611 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:54.850587    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fqpqr"]
Jul 17 10:08:54.898022 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:54.897998    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18a98949-fec9-4b12-a6b0-664275904a47" path="/var/lib/kubelet/pods/18a98949-fec9-4b12-a6b0-664275904a47/volumes"
Jul 17 10:08:58.247947 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.247910    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wm8nq"]
Jul 17 10:08:58.248396 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.248372    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18a98949-fec9-4b12-a6b0-664275904a47" containerName="revoke-keys"
Jul 17 10:08:58.248396 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.248398    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="18a98949-fec9-4b12-a6b0-664275904a47" containerName="revoke-keys"
Jul 17 10:08:58.248488 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.248410    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fe01889-9a26-40c7-8a9e-4213f1cba8a1" containerName="revoke-keys"
Jul 17 10:08:58.248488 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.248415    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fe01889-9a26-40c7-8a9e-4213f1cba8a1" containerName="revoke-keys"
Jul 17 10:08:58.248488 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.248421    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ab65f52-472b-4963-be7c-f721e3c51153" containerName="revoke-keys"
Jul 17 10:08:58.248488 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.248427    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ab65f52-472b-4963-be7c-f721e3c51153" containerName="revoke-keys"
Jul 17 10:08:58.248488 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.248432    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fe01889-9a26-40c7-8a9e-4213f1cba8a1" containerName="revoke-keys"
Jul 17 10:08:58.248488 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.248437    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fe01889-9a26-40c7-8a9e-4213f1cba8a1" containerName="revoke-keys"
Jul 17 10:08:58.248488 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.248442    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18a98949-fec9-4b12-a6b0-664275904a47" containerName="revoke-keys"
Jul 17 10:08:58.248488 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.248447    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="18a98949-fec9-4b12-a6b0-664275904a47" containerName="revoke-keys"
Jul 17 10:08:58.248488 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.248454    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ab65f52-472b-4963-be7c-f721e3c51153" containerName="revoke-keys"
Jul 17 10:08:58.248488 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.248460    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ab65f52-472b-4963-be7c-f721e3c51153" containerName="revoke-keys"
Jul 17 10:08:58.248488 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.248475    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ab65f52-472b-4963-be7c-f721e3c51153" containerName="revoke-keys"
Jul 17 10:08:58.248488 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.248479    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ab65f52-472b-4963-be7c-f721e3c51153" containerName="revoke-keys"
Jul 17 10:08:58.248848 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.248547    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ab65f52-472b-4963-be7c-f721e3c51153" containerName="revoke-keys"
Jul 17 10:08:58.248848 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.248554    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="18a98949-fec9-4b12-a6b0-664275904a47" containerName="revoke-keys"
Jul 17 10:08:58.248848 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.248562    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ab65f52-472b-4963-be7c-f721e3c51153" containerName="revoke-keys"
Jul 17 10:08:58.248848 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.248567    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="18a98949-fec9-4b12-a6b0-664275904a47" containerName="revoke-keys"
Jul 17 10:08:58.248848 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.248574    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="18a98949-fec9-4b12-a6b0-664275904a47" containerName="revoke-keys"
Jul 17 10:08:58.248848 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.248580    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ab65f52-472b-4963-be7c-f721e3c51153" containerName="revoke-keys"
Jul 17 10:08:58.248848 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.248586    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fe01889-9a26-40c7-8a9e-4213f1cba8a1" containerName="revoke-keys"
Jul 17 10:08:58.248848 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.248593    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fe01889-9a26-40c7-8a9e-4213f1cba8a1" containerName="revoke-keys"
Jul 17 10:08:58.248848 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.248598    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fe01889-9a26-40c7-8a9e-4213f1cba8a1" containerName="revoke-keys"
Jul 17 10:08:58.253218 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.253200    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wm8nq"
Jul 17 10:08:58.256382 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.256359    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:08:58.260289 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.260263    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wm8nq"]
Jul 17 10:08:58.372843 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.372807    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de76cc58-cd6a-4c70-bf10-5c83c4fbc653-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-wm8nq\" (UID: \"de76cc58-cd6a-4c70-bf10-5c83c4fbc653\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wm8nq"
Jul 17 10:08:58.474431 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.474393    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de76cc58-cd6a-4c70-bf10-5c83c4fbc653-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-wm8nq\" (UID: \"de76cc58-cd6a-4c70-bf10-5c83c4fbc653\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wm8nq"
Jul 17 10:08:58.475005 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.474985    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de76cc58-cd6a-4c70-bf10-5c83c4fbc653-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-wm8nq\" (UID: \"de76cc58-cd6a-4c70-bf10-5c83c4fbc653\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wm8nq"
Jul 17 10:08:58.564941 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.564839    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wm8nq"
Jul 17 10:08:58.710046 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.710021    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wm8nq"]
Jul 17 10:08:58.712797 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:08:58.712766    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde76cc58_cd6a_4c70_bf10_5c83c4fbc653.slice/crio-cee04e6947ace36349d904f5679541b5b8a9e91606f4a41f7f18bf2aba12e788 WatchSource:0}: Error finding container cee04e6947ace36349d904f5679541b5b8a9e91606f4a41f7f18bf2aba12e788: Status 404 returned error can't find the container with id cee04e6947ace36349d904f5679541b5b8a9e91606f4a41f7f18bf2aba12e788
Jul 17 10:08:58.838160 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:58.838132    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wm8nq" event={"ID":"de76cc58-cd6a-4c70-bf10-5c83c4fbc653","Type":"ContainerStarted","Data":"cee04e6947ace36349d904f5679541b5b8a9e91606f4a41f7f18bf2aba12e788"}
Jul 17 10:08:59.844827 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:59.844788    2574 generic.go:356] "Generic (PLEG): container finished" podID="de76cc58-cd6a-4c70-bf10-5c83c4fbc653" containerID="0a6183afc42ec993ddd7fcf123b57f8f82c819ca2370922dde14dacfc5b7f206" exitCode=6
Jul 17 10:08:59.845220 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:59.844871    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wm8nq" event={"ID":"de76cc58-cd6a-4c70-bf10-5c83c4fbc653","Type":"ContainerDied","Data":"0a6183afc42ec993ddd7fcf123b57f8f82c819ca2370922dde14dacfc5b7f206"}
Jul 17 10:08:59.845220 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:08:59.845018    2574 scope.go:117] "RemoveContainer" containerID="0a6183afc42ec993ddd7fcf123b57f8f82c819ca2370922dde14dacfc5b7f206"
Jul 17 10:09:00.851836 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:00.851800    2574 generic.go:356] "Generic (PLEG): container finished" podID="de76cc58-cd6a-4c70-bf10-5c83c4fbc653" containerID="d437d3c2a83a01259ba2ee84f459cc44b4918d393cd7a5e245a8578c3f89f11a" exitCode=6
Jul 17 10:09:00.852287 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:00.851860    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wm8nq" event={"ID":"de76cc58-cd6a-4c70-bf10-5c83c4fbc653","Type":"ContainerDied","Data":"d437d3c2a83a01259ba2ee84f459cc44b4918d393cd7a5e245a8578c3f89f11a"}
Jul 17 10:09:00.852287 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:00.851909    2574 scope.go:117] "RemoveContainer" containerID="0a6183afc42ec993ddd7fcf123b57f8f82c819ca2370922dde14dacfc5b7f206"
Jul 17 10:09:00.852287 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:00.852064    2574 scope.go:117] "RemoveContainer" containerID="d437d3c2a83a01259ba2ee84f459cc44b4918d393cd7a5e245a8578c3f89f11a"
Jul 17 10:09:00.852412 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:09:00.852341    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-wm8nq_odh-ai-gateway-infra(de76cc58-cd6a-4c70-bf10-5c83c4fbc653)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wm8nq" podUID="de76cc58-cd6a-4c70-bf10-5c83c4fbc653"
Jul 17 10:09:01.859374 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:01.859345    2574 scope.go:117] "RemoveContainer" containerID="d437d3c2a83a01259ba2ee84f459cc44b4918d393cd7a5e245a8578c3f89f11a"
Jul 17 10:09:01.859880 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:09:01.859617    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-wm8nq_odh-ai-gateway-infra(de76cc58-cd6a-4c70-bf10-5c83c4fbc653)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wm8nq" podUID="de76cc58-cd6a-4c70-bf10-5c83c4fbc653"
Jul 17 10:09:13.891410 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:13.891376    2574 scope.go:117] "RemoveContainer" containerID="d437d3c2a83a01259ba2ee84f459cc44b4918d393cd7a5e245a8578c3f89f11a"
Jul 17 10:09:14.935049 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:14.935017    2574 generic.go:356] "Generic (PLEG): container finished" podID="de76cc58-cd6a-4c70-bf10-5c83c4fbc653" containerID="252260471fcd15203bf227fe726206465a846428459a0ee78008866beb8d5ab8" exitCode=6
Jul 17 10:09:14.935558 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:14.935093    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wm8nq" event={"ID":"de76cc58-cd6a-4c70-bf10-5c83c4fbc653","Type":"ContainerDied","Data":"252260471fcd15203bf227fe726206465a846428459a0ee78008866beb8d5ab8"}
Jul 17 10:09:14.935558 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:14.935162    2574 scope.go:117] "RemoveContainer" containerID="d437d3c2a83a01259ba2ee84f459cc44b4918d393cd7a5e245a8578c3f89f11a"
Jul 17 10:09:14.935558 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:14.935550    2574 scope.go:117] "RemoveContainer" containerID="252260471fcd15203bf227fe726206465a846428459a0ee78008866beb8d5ab8"
Jul 17 10:09:14.935805 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:09:14.935784    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-wm8nq_odh-ai-gateway-infra(de76cc58-cd6a-4c70-bf10-5c83c4fbc653)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wm8nq" podUID="de76cc58-cd6a-4c70-bf10-5c83c4fbc653"
Jul 17 10:09:15.963009 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:15.962976    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wm8nq"]
Jul 17 10:09:16.112551 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:16.112519    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wm8nq"
Jul 17 10:09:16.140911 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:16.140877    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v22p7"]
Jul 17 10:09:16.141622 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:16.141597    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de76cc58-cd6a-4c70-bf10-5c83c4fbc653" containerName="revoke-keys"
Jul 17 10:09:16.141622 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:16.141623    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="de76cc58-cd6a-4c70-bf10-5c83c4fbc653" containerName="revoke-keys"
Jul 17 10:09:16.141780 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:16.141637    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de76cc58-cd6a-4c70-bf10-5c83c4fbc653" containerName="revoke-keys"
Jul 17 10:09:16.141780 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:16.141646    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="de76cc58-cd6a-4c70-bf10-5c83c4fbc653" containerName="revoke-keys"
Jul 17 10:09:16.141780 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:16.141663    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fe01889-9a26-40c7-8a9e-4213f1cba8a1" containerName="revoke-keys"
Jul 17 10:09:16.141780 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:16.141672    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fe01889-9a26-40c7-8a9e-4213f1cba8a1" containerName="revoke-keys"
Jul 17 10:09:16.141780 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:16.141683    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18a98949-fec9-4b12-a6b0-664275904a47" containerName="revoke-keys"
Jul 17 10:09:16.141780 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:16.141692    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="18a98949-fec9-4b12-a6b0-664275904a47" containerName="revoke-keys"
Jul 17 10:09:16.141978 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:16.141838    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="de76cc58-cd6a-4c70-bf10-5c83c4fbc653" containerName="revoke-keys"
Jul 17 10:09:16.141978 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:16.141858    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="de76cc58-cd6a-4c70-bf10-5c83c4fbc653" containerName="revoke-keys"
Jul 17 10:09:16.145325 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:16.145302    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v22p7"
Jul 17 10:09:16.145523 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:16.145503    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de76cc58-cd6a-4c70-bf10-5c83c4fbc653-maas-api-service-ca\") pod \"de76cc58-cd6a-4c70-bf10-5c83c4fbc653\" (UID: \"de76cc58-cd6a-4c70-bf10-5c83c4fbc653\") "
Jul 17 10:09:16.145885 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:16.145860    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de76cc58-cd6a-4c70-bf10-5c83c4fbc653-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "de76cc58-cd6a-4c70-bf10-5c83c4fbc653" (UID: "de76cc58-cd6a-4c70-bf10-5c83c4fbc653"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:09:16.150186 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:16.150162    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v22p7"]
Jul 17 10:09:16.246278 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:16.246146    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-v22p7\" (UID: \"43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v22p7"
Jul 17 10:09:16.246452 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:16.246360    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de76cc58-cd6a-4c70-bf10-5c83c4fbc653-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:09:16.347643 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:16.347604    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-v22p7\" (UID: \"43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v22p7"
Jul 17 10:09:16.348250 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:16.348213    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-v22p7\" (UID: \"43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v22p7"
Jul 17 10:09:16.458378 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:16.458344    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v22p7"
Jul 17 10:09:16.805458 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:16.805426    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v22p7"]
Jul 17 10:09:16.807109 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:09:16.807065    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43db0699_2c1f_4c64_84ba_0f1bbc7b3e0e.slice/crio-cc164fb471b8486aed3659583e484f090d5f0f581bda2a8cb8331ea0c43fa0e7 WatchSource:0}: Error finding container cc164fb471b8486aed3659583e484f090d5f0f581bda2a8cb8331ea0c43fa0e7: Status 404 returned error can't find the container with id cc164fb471b8486aed3659583e484f090d5f0f581bda2a8cb8331ea0c43fa0e7
Jul 17 10:09:16.948975 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:16.948936    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v22p7" event={"ID":"43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e","Type":"ContainerStarted","Data":"cc164fb471b8486aed3659583e484f090d5f0f581bda2a8cb8331ea0c43fa0e7"}
Jul 17 10:09:16.950203 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:16.950178    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wm8nq" event={"ID":"de76cc58-cd6a-4c70-bf10-5c83c4fbc653","Type":"ContainerDied","Data":"cee04e6947ace36349d904f5679541b5b8a9e91606f4a41f7f18bf2aba12e788"}
Jul 17 10:09:16.950361 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:16.950230    2574 scope.go:117] "RemoveContainer" containerID="252260471fcd15203bf227fe726206465a846428459a0ee78008866beb8d5ab8"
Jul 17 10:09:16.950361 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:16.950188    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wm8nq"
Jul 17 10:09:16.968532 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:16.968507    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wm8nq"]
Jul 17 10:09:16.976266 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:16.972776    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wm8nq"]
Jul 17 10:09:17.168471 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:17.168425    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-xn89z"]
Jul 17 10:09:17.168934 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:17.168920    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de76cc58-cd6a-4c70-bf10-5c83c4fbc653" containerName="revoke-keys"
Jul 17 10:09:17.168984 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:17.168936    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="de76cc58-cd6a-4c70-bf10-5c83c4fbc653" containerName="revoke-keys"
Jul 17 10:09:17.169020 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:17.169003    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="de76cc58-cd6a-4c70-bf10-5c83c4fbc653" containerName="revoke-keys"
Jul 17 10:09:17.172161 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:17.172145    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-xn89z"
Jul 17 10:09:17.180787 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:17.180759    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-xn89z"]
Jul 17 10:09:17.256168 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:17.256125    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7788a41a-ffe5-40fa-af6a-6f6269b71c20-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-xn89z\" (UID: \"7788a41a-ffe5-40fa-af6a-6f6269b71c20\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-xn89z"
Jul 17 10:09:17.357744 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:17.357701    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7788a41a-ffe5-40fa-af6a-6f6269b71c20-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-xn89z\" (UID: \"7788a41a-ffe5-40fa-af6a-6f6269b71c20\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-xn89z"
Jul 17 10:09:17.358337 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:17.358319    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7788a41a-ffe5-40fa-af6a-6f6269b71c20-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-xn89z\" (UID: \"7788a41a-ffe5-40fa-af6a-6f6269b71c20\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-xn89z"
Jul 17 10:09:17.485522 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:17.485482    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-xn89z"
Jul 17 10:09:17.640739 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:17.640601    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-xn89z"]
Jul 17 10:09:17.658121 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:09:17.658088    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7788a41a_ffe5_40fa_af6a_6f6269b71c20.slice/crio-e20dcb63708bc64a6f501dd906126b2cf48b72a793b6e14e9ea2061eed4a9490 WatchSource:0}: Error finding container e20dcb63708bc64a6f501dd906126b2cf48b72a793b6e14e9ea2061eed4a9490: Status 404 returned error can't find the container with id e20dcb63708bc64a6f501dd906126b2cf48b72a793b6e14e9ea2061eed4a9490
Jul 17 10:09:17.962158 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:17.962123    2574 generic.go:356] "Generic (PLEG): container finished" podID="43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e" containerID="9d6e4dd0fdd046f64c17046931b6949048a1001d7c12d308fdd9fdfad04bbd09" exitCode=6
Jul 17 10:09:17.962390 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:17.962210    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v22p7" event={"ID":"43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e","Type":"ContainerDied","Data":"9d6e4dd0fdd046f64c17046931b6949048a1001d7c12d308fdd9fdfad04bbd09"}
Jul 17 10:09:17.962480 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:17.962461    2574 scope.go:117] "RemoveContainer" containerID="9d6e4dd0fdd046f64c17046931b6949048a1001d7c12d308fdd9fdfad04bbd09"
Jul 17 10:09:17.964292 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:17.964265    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-xn89z" event={"ID":"7788a41a-ffe5-40fa-af6a-6f6269b71c20","Type":"ContainerStarted","Data":"e20dcb63708bc64a6f501dd906126b2cf48b72a793b6e14e9ea2061eed4a9490"}
Jul 17 10:09:18.898737 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:18.898703    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de76cc58-cd6a-4c70-bf10-5c83c4fbc653" path="/var/lib/kubelet/pods/de76cc58-cd6a-4c70-bf10-5c83c4fbc653/volumes"
Jul 17 10:09:18.971764 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:18.971686    2574 generic.go:356] "Generic (PLEG): container finished" podID="43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e" containerID="04dc2cff1e43e4baf5a2723b400716440ff9277056152d02864032f8fc8f8303" exitCode=6
Jul 17 10:09:18.971930 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:18.971781    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v22p7" event={"ID":"43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e","Type":"ContainerDied","Data":"04dc2cff1e43e4baf5a2723b400716440ff9277056152d02864032f8fc8f8303"}
Jul 17 10:09:18.971930 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:18.971860    2574 scope.go:117] "RemoveContainer" containerID="9d6e4dd0fdd046f64c17046931b6949048a1001d7c12d308fdd9fdfad04bbd09"
Jul 17 10:09:18.972037 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:18.971957    2574 scope.go:117] "RemoveContainer" containerID="04dc2cff1e43e4baf5a2723b400716440ff9277056152d02864032f8fc8f8303"
Jul 17 10:09:18.972200 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:09:18.972168    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-v22p7_odh-ai-gateway-infra(43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v22p7" podUID="43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e"
Jul 17 10:09:18.973340 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:18.973318    2574 generic.go:356] "Generic (PLEG): container finished" podID="7788a41a-ffe5-40fa-af6a-6f6269b71c20" containerID="5cedbacaf243afe3ee7e5ee250cfcaec2fa752bd8aae5b0014130b3f2c81e34b" exitCode=6
Jul 17 10:09:18.973431 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:18.973394    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-xn89z" event={"ID":"7788a41a-ffe5-40fa-af6a-6f6269b71c20","Type":"ContainerDied","Data":"5cedbacaf243afe3ee7e5ee250cfcaec2fa752bd8aae5b0014130b3f2c81e34b"}
Jul 17 10:09:18.973536 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:18.973523    2574 scope.go:117] "RemoveContainer" containerID="5cedbacaf243afe3ee7e5ee250cfcaec2fa752bd8aae5b0014130b3f2c81e34b"
Jul 17 10:09:19.980911 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:19.980816    2574 scope.go:117] "RemoveContainer" containerID="04dc2cff1e43e4baf5a2723b400716440ff9277056152d02864032f8fc8f8303"
Jul 17 10:09:19.981335 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:09:19.981029    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-v22p7_odh-ai-gateway-infra(43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v22p7" podUID="43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e"
Jul 17 10:09:19.982525 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:19.982498    2574 generic.go:356] "Generic (PLEG): container finished" podID="7788a41a-ffe5-40fa-af6a-6f6269b71c20" containerID="d23c2c624a1bd8d0892a99361519a78d52fb4237b8ccb2356347660e78470f35" exitCode=6
Jul 17 10:09:19.982634 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:19.982536    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-xn89z" event={"ID":"7788a41a-ffe5-40fa-af6a-6f6269b71c20","Type":"ContainerDied","Data":"d23c2c624a1bd8d0892a99361519a78d52fb4237b8ccb2356347660e78470f35"}
Jul 17 10:09:19.982634 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:19.982592    2574 scope.go:117] "RemoveContainer" containerID="5cedbacaf243afe3ee7e5ee250cfcaec2fa752bd8aae5b0014130b3f2c81e34b"
Jul 17 10:09:19.982842 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:19.982823    2574 scope.go:117] "RemoveContainer" containerID="d23c2c624a1bd8d0892a99361519a78d52fb4237b8ccb2356347660e78470f35"
Jul 17 10:09:19.983029 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:09:19.983010    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-xn89z_odh-ai-gateway-infra(7788a41a-ffe5-40fa-af6a-6f6269b71c20)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-xn89z" podUID="7788a41a-ffe5-40fa-af6a-6f6269b71c20"
Jul 17 10:09:20.990145 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:20.990118    2574 scope.go:117] "RemoveContainer" containerID="d23c2c624a1bd8d0892a99361519a78d52fb4237b8ccb2356347660e78470f35"
Jul 17 10:09:20.990552 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:09:20.990332    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-xn89z_odh-ai-gateway-infra(7788a41a-ffe5-40fa-af6a-6f6269b71c20)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-xn89z" podUID="7788a41a-ffe5-40fa-af6a-6f6269b71c20"
Jul 17 10:09:25.554719 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:25.554683    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-twmgc"]
Jul 17 10:09:25.558613 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:25.558596    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-twmgc"
Jul 17 10:09:25.567182 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:25.567124    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-twmgc"]
Jul 17 10:09:25.634626 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:25.634588    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-twmgc\" (UID: \"320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-twmgc"
Jul 17 10:09:25.735660 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:25.735625    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-twmgc\" (UID: \"320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-twmgc"
Jul 17 10:09:25.736201 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:25.736181    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-twmgc\" (UID: \"320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-twmgc"
Jul 17 10:09:25.870619 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:25.870527    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-twmgc"
Jul 17 10:09:26.016684 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:26.016662    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-twmgc"]
Jul 17 10:09:26.018994 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:09:26.018962    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod320ca2f9_f3c3_4e12_9e0f_953f80b2e7d3.slice/crio-e1f2cd954ded3207e6698c495a0c4addfc592bd5e85af95a7347135afb3dfdcd WatchSource:0}: Error finding container e1f2cd954ded3207e6698c495a0c4addfc592bd5e85af95a7347135afb3dfdcd: Status 404 returned error can't find the container with id e1f2cd954ded3207e6698c495a0c4addfc592bd5e85af95a7347135afb3dfdcd
Jul 17 10:09:27.027592 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:27.027493    2574 generic.go:356] "Generic (PLEG): container finished" podID="320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3" containerID="172b6dbc03735c4c0664a09547c16b8618c002b3cbc26df9d2091c303af88c91" exitCode=6
Jul 17 10:09:27.027592 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:27.027543    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-twmgc" event={"ID":"320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3","Type":"ContainerDied","Data":"172b6dbc03735c4c0664a09547c16b8618c002b3cbc26df9d2091c303af88c91"}
Jul 17 10:09:27.027994 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:27.027600    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-twmgc" event={"ID":"320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3","Type":"ContainerStarted","Data":"e1f2cd954ded3207e6698c495a0c4addfc592bd5e85af95a7347135afb3dfdcd"}
Jul 17 10:09:27.027994 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:27.027824    2574 scope.go:117] "RemoveContainer" containerID="172b6dbc03735c4c0664a09547c16b8618c002b3cbc26df9d2091c303af88c91"
Jul 17 10:09:28.034848 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:28.034817    2574 generic.go:356] "Generic (PLEG): container finished" podID="320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3" containerID="0d5851878e9f9d0814f9acbe94c4421f1dec552a477e2164de973e217221d8b7" exitCode=6
Jul 17 10:09:28.035271 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:28.034901    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-twmgc" event={"ID":"320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3","Type":"ContainerDied","Data":"0d5851878e9f9d0814f9acbe94c4421f1dec552a477e2164de973e217221d8b7"}
Jul 17 10:09:28.035271 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:28.034969    2574 scope.go:117] "RemoveContainer" containerID="172b6dbc03735c4c0664a09547c16b8618c002b3cbc26df9d2091c303af88c91"
Jul 17 10:09:28.035271 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:28.035111    2574 scope.go:117] "RemoveContainer" containerID="0d5851878e9f9d0814f9acbe94c4421f1dec552a477e2164de973e217221d8b7"
Jul 17 10:09:28.035408 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:09:28.035387    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-twmgc_odh-ai-gateway-infra(320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-twmgc" podUID="320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3"
Jul 17 10:09:29.042871 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:29.042839    2574 scope.go:117] "RemoveContainer" containerID="0d5851878e9f9d0814f9acbe94c4421f1dec552a477e2164de973e217221d8b7"
Jul 17 10:09:29.043382 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:09:29.043103    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-twmgc_odh-ai-gateway-infra(320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-twmgc" podUID="320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3"
Jul 17 10:09:33.891199 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:33.891165    2574 scope.go:117] "RemoveContainer" containerID="04dc2cff1e43e4baf5a2723b400716440ff9277056152d02864032f8fc8f8303"
Jul 17 10:09:34.892112 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:34.892034    2574 scope.go:117] "RemoveContainer" containerID="d23c2c624a1bd8d0892a99361519a78d52fb4237b8ccb2356347660e78470f35"
Jul 17 10:09:35.083414 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:35.083376    2574 generic.go:356] "Generic (PLEG): container finished" podID="43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e" containerID="cdd249a231ceefb7e765417944f813636d26839a0260e7c3b841ac5c2ff979ae" exitCode=6
Jul 17 10:09:35.083575 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:35.083451    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v22p7" event={"ID":"43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e","Type":"ContainerDied","Data":"cdd249a231ceefb7e765417944f813636d26839a0260e7c3b841ac5c2ff979ae"}
Jul 17 10:09:35.083575 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:35.083515    2574 scope.go:117] "RemoveContainer" containerID="04dc2cff1e43e4baf5a2723b400716440ff9277056152d02864032f8fc8f8303"
Jul 17 10:09:35.083905 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:35.083888    2574 scope.go:117] "RemoveContainer" containerID="cdd249a231ceefb7e765417944f813636d26839a0260e7c3b841ac5c2ff979ae"
Jul 17 10:09:35.084148 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:09:35.084113    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-v22p7_odh-ai-gateway-infra(43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v22p7" podUID="43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e"
Jul 17 10:09:36.092902 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:36.092805    2574 generic.go:356] "Generic (PLEG): container finished" podID="7788a41a-ffe5-40fa-af6a-6f6269b71c20" containerID="3e0bc7a615384fdfb56476b80bdf52c8826455e9f12a09f0a9d445828a7f6d3b" exitCode=6
Jul 17 10:09:36.093395 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:36.092887    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-xn89z" event={"ID":"7788a41a-ffe5-40fa-af6a-6f6269b71c20","Type":"ContainerDied","Data":"3e0bc7a615384fdfb56476b80bdf52c8826455e9f12a09f0a9d445828a7f6d3b"}
Jul 17 10:09:36.093395 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:36.093013    2574 scope.go:117] "RemoveContainer" containerID="d23c2c624a1bd8d0892a99361519a78d52fb4237b8ccb2356347660e78470f35"
Jul 17 10:09:36.093395 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:36.093390    2574 scope.go:117] "RemoveContainer" containerID="3e0bc7a615384fdfb56476b80bdf52c8826455e9f12a09f0a9d445828a7f6d3b"
Jul 17 10:09:36.093671 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:09:36.093647    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-xn89z_odh-ai-gateway-infra(7788a41a-ffe5-40fa-af6a-6f6269b71c20)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-xn89z" podUID="7788a41a-ffe5-40fa-af6a-6f6269b71c20"
Jul 17 10:09:36.109912 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:36.109888    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v22p7"]
Jul 17 10:09:36.264834 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:36.264806    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v22p7"
Jul 17 10:09:36.339913 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:36.339880    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e-maas-api-service-ca\") pod \"43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e\" (UID: \"43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e\") "
Jul 17 10:09:36.340286 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:36.340226    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e" (UID: "43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:09:36.441077 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:36.441043    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:09:37.101458 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:37.101423    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v22p7"
Jul 17 10:09:37.101935 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:37.101458    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v22p7" event={"ID":"43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e","Type":"ContainerDied","Data":"cc164fb471b8486aed3659583e484f090d5f0f581bda2a8cb8331ea0c43fa0e7"}
Jul 17 10:09:37.101935 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:37.101536    2574 scope.go:117] "RemoveContainer" containerID="cdd249a231ceefb7e765417944f813636d26839a0260e7c3b841ac5c2ff979ae"
Jul 17 10:09:37.121041 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:37.121014    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-xn89z"]
Jul 17 10:09:37.123693 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:37.123673    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v22p7"]
Jul 17 10:09:37.130779 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:37.130755    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v22p7"]
Jul 17 10:09:37.266450 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:37.266428    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-xn89z"
Jul 17 10:09:37.350551 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:37.350521    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7788a41a-ffe5-40fa-af6a-6f6269b71c20-maas-api-service-ca\") pod \"7788a41a-ffe5-40fa-af6a-6f6269b71c20\" (UID: \"7788a41a-ffe5-40fa-af6a-6f6269b71c20\") "
Jul 17 10:09:37.350893 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:37.350868    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7788a41a-ffe5-40fa-af6a-6f6269b71c20-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7788a41a-ffe5-40fa-af6a-6f6269b71c20" (UID: "7788a41a-ffe5-40fa-af6a-6f6269b71c20"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:09:37.452318 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:37.452210    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7788a41a-ffe5-40fa-af6a-6f6269b71c20-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:09:38.111810 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:38.111780    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-xn89z"
Jul 17 10:09:38.112208 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:38.111784    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-xn89z" event={"ID":"7788a41a-ffe5-40fa-af6a-6f6269b71c20","Type":"ContainerDied","Data":"e20dcb63708bc64a6f501dd906126b2cf48b72a793b6e14e9ea2061eed4a9490"}
Jul 17 10:09:38.112208 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:38.111915    2574 scope.go:117] "RemoveContainer" containerID="3e0bc7a615384fdfb56476b80bdf52c8826455e9f12a09f0a9d445828a7f6d3b"
Jul 17 10:09:38.134320 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:38.134287    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-xn89z"]
Jul 17 10:09:38.139378 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:38.139357    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-xn89z"]
Jul 17 10:09:38.906038 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:38.905998    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e" path="/var/lib/kubelet/pods/43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e/volumes"
Jul 17 10:09:38.906517 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:38.906500    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7788a41a-ffe5-40fa-af6a-6f6269b71c20" path="/var/lib/kubelet/pods/7788a41a-ffe5-40fa-af6a-6f6269b71c20/volumes"
Jul 17 10:09:41.891916 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:41.891883    2574 scope.go:117] "RemoveContainer" containerID="0d5851878e9f9d0814f9acbe94c4421f1dec552a477e2164de973e217221d8b7"
Jul 17 10:09:43.141048 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:43.141016    2574 generic.go:356] "Generic (PLEG): container finished" podID="320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3" containerID="08e4d3bd29c71fda980a346506061410fadb6aff90b4113af077ad2af3d7e705" exitCode=6
Jul 17 10:09:43.141512 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:43.141099    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-twmgc" event={"ID":"320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3","Type":"ContainerDied","Data":"08e4d3bd29c71fda980a346506061410fadb6aff90b4113af077ad2af3d7e705"}
Jul 17 10:09:43.141512 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:43.141157    2574 scope.go:117] "RemoveContainer" containerID="0d5851878e9f9d0814f9acbe94c4421f1dec552a477e2164de973e217221d8b7"
Jul 17 10:09:43.141602 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:43.141571    2574 scope.go:117] "RemoveContainer" containerID="08e4d3bd29c71fda980a346506061410fadb6aff90b4113af077ad2af3d7e705"
Jul 17 10:09:43.141854 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:09:43.141823    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-twmgc_odh-ai-gateway-infra(320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-twmgc" podUID="320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3"
Jul 17 10:09:44.168083 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:44.168052    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-twmgc"]
Jul 17 10:09:44.319160 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:44.319138    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-twmgc"
Jul 17 10:09:44.419440 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:44.419357    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3-maas-api-service-ca\") pod \"320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3\" (UID: \"320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3\") "
Jul 17 10:09:44.419690 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:44.419669    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3" (UID: "320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:09:44.520503 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:44.520468    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:09:45.155156 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:45.155128    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-twmgc"
Jul 17 10:09:45.155353 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:45.155159    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-twmgc" event={"ID":"320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3","Type":"ContainerDied","Data":"e1f2cd954ded3207e6698c495a0c4addfc592bd5e85af95a7347135afb3dfdcd"}
Jul 17 10:09:45.155353 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:45.155219    2574 scope.go:117] "RemoveContainer" containerID="08e4d3bd29c71fda980a346506061410fadb6aff90b4113af077ad2af3d7e705"
Jul 17 10:09:45.172861 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:45.172837    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-twmgc"]
Jul 17 10:09:45.177953 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:45.177931    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-twmgc"]
Jul 17 10:09:46.898581 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:46.898555    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3" path="/var/lib/kubelet/pods/320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3/volumes"
Jul 17 10:09:48.305382 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.305345    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-2jts7"]
Jul 17 10:09:48.305988 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.305968    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e" containerName="revoke-keys"
Jul 17 10:09:48.306082 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.305991    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e" containerName="revoke-keys"
Jul 17 10:09:48.306082 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.306008    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7788a41a-ffe5-40fa-af6a-6f6269b71c20" containerName="revoke-keys"
Jul 17 10:09:48.306082 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.306016    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7788a41a-ffe5-40fa-af6a-6f6269b71c20" containerName="revoke-keys"
Jul 17 10:09:48.306082 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.306034    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e" containerName="revoke-keys"
Jul 17 10:09:48.306082 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.306043    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e" containerName="revoke-keys"
Jul 17 10:09:48.306082 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.306056    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7788a41a-ffe5-40fa-af6a-6f6269b71c20" containerName="revoke-keys"
Jul 17 10:09:48.306082 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.306064    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7788a41a-ffe5-40fa-af6a-6f6269b71c20" containerName="revoke-keys"
Jul 17 10:09:48.306082 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.306073    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3" containerName="revoke-keys"
Jul 17 10:09:48.306082 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.306081    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3" containerName="revoke-keys"
Jul 17 10:09:48.306538 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.306090    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7788a41a-ffe5-40fa-af6a-6f6269b71c20" containerName="revoke-keys"
Jul 17 10:09:48.306538 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.306098    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="7788a41a-ffe5-40fa-af6a-6f6269b71c20" containerName="revoke-keys"
Jul 17 10:09:48.306538 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.306111    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e" containerName="revoke-keys"
Jul 17 10:09:48.306538 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.306120    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e" containerName="revoke-keys"
Jul 17 10:09:48.306538 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.306145    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3" containerName="revoke-keys"
Jul 17 10:09:48.306538 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.306154    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3" containerName="revoke-keys"
Jul 17 10:09:48.306538 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.306162    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3" containerName="revoke-keys"
Jul 17 10:09:48.306538 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.306170    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3" containerName="revoke-keys"
Jul 17 10:09:48.306538 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.306298    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e" containerName="revoke-keys"
Jul 17 10:09:48.306538 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.306315    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3" containerName="revoke-keys"
Jul 17 10:09:48.306538 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.306327    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7788a41a-ffe5-40fa-af6a-6f6269b71c20" containerName="revoke-keys"
Jul 17 10:09:48.306538 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.306339    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3" containerName="revoke-keys"
Jul 17 10:09:48.306538 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.306349    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="320ca2f9-f3c3-4e12-9e0f-953f80b2e7d3" containerName="revoke-keys"
Jul 17 10:09:48.306538 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.306359    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e" containerName="revoke-keys"
Jul 17 10:09:48.306538 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.306369    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7788a41a-ffe5-40fa-af6a-6f6269b71c20" containerName="revoke-keys"
Jul 17 10:09:48.310947 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.310925    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-2jts7"
Jul 17 10:09:48.315274 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.315256    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:09:48.319903 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.319880    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-2jts7"]
Jul 17 10:09:48.457265 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.457194    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36a00ba7-3c15-48c9-a5b6-47aec1d29bc1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-2jts7\" (UID: \"36a00ba7-3c15-48c9-a5b6-47aec1d29bc1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-2jts7"
Jul 17 10:09:48.558649 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.558565    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36a00ba7-3c15-48c9-a5b6-47aec1d29bc1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-2jts7\" (UID: \"36a00ba7-3c15-48c9-a5b6-47aec1d29bc1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-2jts7"
Jul 17 10:09:48.559185 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.559165    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36a00ba7-3c15-48c9-a5b6-47aec1d29bc1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-2jts7\" (UID: \"36a00ba7-3c15-48c9-a5b6-47aec1d29bc1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-2jts7"
Jul 17 10:09:48.622449 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.622416    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-2jts7"
Jul 17 10:09:48.973432 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:48.973398    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-2jts7"]
Jul 17 10:09:48.975294 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:09:48.975266    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36a00ba7_3c15_48c9_a5b6_47aec1d29bc1.slice/crio-d8fe441781cb9f5589c3884a7eee99eec42ee8d4cc22888f3c01e39dd66dd02b WatchSource:0}: Error finding container d8fe441781cb9f5589c3884a7eee99eec42ee8d4cc22888f3c01e39dd66dd02b: Status 404 returned error can't find the container with id d8fe441781cb9f5589c3884a7eee99eec42ee8d4cc22888f3c01e39dd66dd02b
Jul 17 10:09:49.179492 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:49.179455    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-2jts7" event={"ID":"36a00ba7-3c15-48c9-a5b6-47aec1d29bc1","Type":"ContainerStarted","Data":"d8fe441781cb9f5589c3884a7eee99eec42ee8d4cc22888f3c01e39dd66dd02b"}
Jul 17 10:09:50.186452 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:50.186413    2574 generic.go:356] "Generic (PLEG): container finished" podID="36a00ba7-3c15-48c9-a5b6-47aec1d29bc1" containerID="7509785fe58e01555e535828b04774f5bb165b92176dd03dbbad55ca737e37d9" exitCode=6
Jul 17 10:09:50.186861 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:50.186508    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-2jts7" event={"ID":"36a00ba7-3c15-48c9-a5b6-47aec1d29bc1","Type":"ContainerDied","Data":"7509785fe58e01555e535828b04774f5bb165b92176dd03dbbad55ca737e37d9"}
Jul 17 10:09:50.186861 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:50.186623    2574 scope.go:117] "RemoveContainer" containerID="7509785fe58e01555e535828b04774f5bb165b92176dd03dbbad55ca737e37d9"
Jul 17 10:09:51.193893 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:51.193861    2574 generic.go:356] "Generic (PLEG): container finished" podID="36a00ba7-3c15-48c9-a5b6-47aec1d29bc1" containerID="52e7f1393f6e3743111b32c787c3995213f237523d39e5526ae623428a23f7de" exitCode=6
Jul 17 10:09:51.194312 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:51.193929    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-2jts7" event={"ID":"36a00ba7-3c15-48c9-a5b6-47aec1d29bc1","Type":"ContainerDied","Data":"52e7f1393f6e3743111b32c787c3995213f237523d39e5526ae623428a23f7de"}
Jul 17 10:09:51.194312 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:51.193977    2574 scope.go:117] "RemoveContainer" containerID="7509785fe58e01555e535828b04774f5bb165b92176dd03dbbad55ca737e37d9"
Jul 17 10:09:51.194312 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:51.194155    2574 scope.go:117] "RemoveContainer" containerID="52e7f1393f6e3743111b32c787c3995213f237523d39e5526ae623428a23f7de"
Jul 17 10:09:51.194454 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:09:51.194424    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-2jts7_odh-ai-gateway-infra(36a00ba7-3c15-48c9-a5b6-47aec1d29bc1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-2jts7" podUID="36a00ba7-3c15-48c9-a5b6-47aec1d29bc1"
Jul 17 10:09:52.201736 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:09:52.201705    2574 scope.go:117] "RemoveContainer" containerID="52e7f1393f6e3743111b32c787c3995213f237523d39e5526ae623428a23f7de"
Jul 17 10:09:52.202148 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:09:52.201975    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-2jts7_odh-ai-gateway-infra(36a00ba7-3c15-48c9-a5b6-47aec1d29bc1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-2jts7" podUID="36a00ba7-3c15-48c9-a5b6-47aec1d29bc1"
Jul 17 10:10:03.891532 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:03.891494    2574 scope.go:117] "RemoveContainer" containerID="52e7f1393f6e3743111b32c787c3995213f237523d39e5526ae623428a23f7de"
Jul 17 10:10:04.147297 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:04.147211    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6bff328f-lk45q"]
Jul 17 10:10:04.147742 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:04.147727    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="7788a41a-ffe5-40fa-af6a-6f6269b71c20" containerName="revoke-keys"
Jul 17 10:10:04.147790 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:04.147746    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="43db0699-2c1f-4c64-84ba-0f1bbc7b3e0e" containerName="revoke-keys"
Jul 17 10:10:04.152369 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:04.152352    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6bff328f-lk45q"
Jul 17 10:10:04.158616 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:04.158572    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6bff328f-lk45q"]
Jul 17 10:10:04.213805 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:04.213773    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d7247e2-690f-4fa1-858a-b6d1c2d98f43-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6bff328f-lk45q\" (UID: \"0d7247e2-690f-4fa1-858a-b6d1c2d98f43\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6bff328f-lk45q"
Jul 17 10:10:04.276665 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:04.276629    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-2jts7" event={"ID":"36a00ba7-3c15-48c9-a5b6-47aec1d29bc1","Type":"ContainerStarted","Data":"4208da9b4ee8f9f0b11942ba213ee2ef1730c8a265a1a06f7fcc66b5019967ae"}
Jul 17 10:10:04.294449 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:04.294403    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-2jts7" podStartSLOduration=16.294389999 podStartE2EDuration="16.294389999s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:09:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:10:04.292208435 +0000 UTC m=+2601.874846810" watchObservedRunningTime="2026-07-17 10:10:04.294389999 +0000 UTC m=+2601.877028375"
Jul 17 10:10:04.314868 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:04.314837    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d7247e2-690f-4fa1-858a-b6d1c2d98f43-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6bff328f-lk45q\" (UID: \"0d7247e2-690f-4fa1-858a-b6d1c2d98f43\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6bff328f-lk45q"
Jul 17 10:10:04.315501 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:04.315480    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d7247e2-690f-4fa1-858a-b6d1c2d98f43-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6bff328f-lk45q\" (UID: \"0d7247e2-690f-4fa1-858a-b6d1c2d98f43\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6bff328f-lk45q"
Jul 17 10:10:04.464375 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:04.464342    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6bff328f-lk45q"
Jul 17 10:10:04.635058 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:04.635026    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6bff328f-lk45q"]
Jul 17 10:10:04.636897 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:10:04.636870    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d7247e2_690f_4fa1_858a_b6d1c2d98f43.slice/crio-f7a87f39f5ab5d4c6a96d3d4aa1002752ff8d14c9be8690881c3ab72c2765ff1 WatchSource:0}: Error finding container f7a87f39f5ab5d4c6a96d3d4aa1002752ff8d14c9be8690881c3ab72c2765ff1: Status 404 returned error can't find the container with id f7a87f39f5ab5d4c6a96d3d4aa1002752ff8d14c9be8690881c3ab72c2765ff1
Jul 17 10:10:05.283295 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:05.283226    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6bff328f-lk45q" event={"ID":"0d7247e2-690f-4fa1-858a-b6d1c2d98f43","Type":"ContainerStarted","Data":"dca6e411786a765d708cfa215a118c7896d919373c4df5df98fe5ed21f118181"}
Jul 17 10:10:05.283776 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:05.283359    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6bff328f-lk45q" event={"ID":"0d7247e2-690f-4fa1-858a-b6d1c2d98f43","Type":"ContainerStarted","Data":"f7a87f39f5ab5d4c6a96d3d4aa1002752ff8d14c9be8690881c3ab72c2765ff1"}
Jul 17 10:10:05.285126 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:05.285101    2574 generic.go:356] "Generic (PLEG): container finished" podID="36a00ba7-3c15-48c9-a5b6-47aec1d29bc1" containerID="4208da9b4ee8f9f0b11942ba213ee2ef1730c8a265a1a06f7fcc66b5019967ae" exitCode=6
Jul 17 10:10:05.285256 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:05.285169    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-2jts7" event={"ID":"36a00ba7-3c15-48c9-a5b6-47aec1d29bc1","Type":"ContainerDied","Data":"4208da9b4ee8f9f0b11942ba213ee2ef1730c8a265a1a06f7fcc66b5019967ae"}
Jul 17 10:10:05.285256 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:05.285220    2574 scope.go:117] "RemoveContainer" containerID="52e7f1393f6e3743111b32c787c3995213f237523d39e5526ae623428a23f7de"
Jul 17 10:10:05.285496 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:05.285475    2574 scope.go:117] "RemoveContainer" containerID="4208da9b4ee8f9f0b11942ba213ee2ef1730c8a265a1a06f7fcc66b5019967ae"
Jul 17 10:10:05.285718 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:10:05.285695    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-2jts7_odh-ai-gateway-infra(36a00ba7-3c15-48c9-a5b6-47aec1d29bc1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-2jts7" podUID="36a00ba7-3c15-48c9-a5b6-47aec1d29bc1"
Jul 17 10:10:05.299879 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:05.299830    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6bff328f-lk45q" podStartSLOduration=1.29981304 podStartE2EDuration="1.29981304s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:10:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:10:05.298978378 +0000 UTC m=+2602.881616756" watchObservedRunningTime="2026-07-17 10:10:05.29981304 +0000 UTC m=+2602.882451416"
Jul 17 10:10:05.306883 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:05.306854    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-2jts7"]
Jul 17 10:10:06.292191 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:06.292153    2574 generic.go:356] "Generic (PLEG): container finished" podID="0d7247e2-690f-4fa1-858a-b6d1c2d98f43" containerID="dca6e411786a765d708cfa215a118c7896d919373c4df5df98fe5ed21f118181" exitCode=0
Jul 17 10:10:06.292684 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:06.292271    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6bff328f-lk45q" event={"ID":"0d7247e2-690f-4fa1-858a-b6d1c2d98f43","Type":"ContainerDied","Data":"dca6e411786a765d708cfa215a118c7896d919373c4df5df98fe5ed21f118181"}
Jul 17 10:10:06.446783 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:06.446761    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-2jts7"
Jul 17 10:10:06.536834 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:06.536801    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36a00ba7-3c15-48c9-a5b6-47aec1d29bc1-maas-api-service-ca\") pod \"36a00ba7-3c15-48c9-a5b6-47aec1d29bc1\" (UID: \"36a00ba7-3c15-48c9-a5b6-47aec1d29bc1\") "
Jul 17 10:10:06.537151 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:06.537126    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36a00ba7-3c15-48c9-a5b6-47aec1d29bc1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "36a00ba7-3c15-48c9-a5b6-47aec1d29bc1" (UID: "36a00ba7-3c15-48c9-a5b6-47aec1d29bc1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:10:06.638128 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:06.638042    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/36a00ba7-3c15-48c9-a5b6-47aec1d29bc1-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:10:07.300412 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:07.300383    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-2jts7"
Jul 17 10:10:07.300871 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:07.300406    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-2jts7" event={"ID":"36a00ba7-3c15-48c9-a5b6-47aec1d29bc1","Type":"ContainerDied","Data":"d8fe441781cb9f5589c3884a7eee99eec42ee8d4cc22888f3c01e39dd66dd02b"}
Jul 17 10:10:07.300871 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:07.300473    2574 scope.go:117] "RemoveContainer" containerID="4208da9b4ee8f9f0b11942ba213ee2ef1730c8a265a1a06f7fcc66b5019967ae"
Jul 17 10:10:07.318909 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:07.318879    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-2jts7"]
Jul 17 10:10:07.326377 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:07.326350    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-2jts7"]
Jul 17 10:10:07.447077 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:07.447046    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6bff328f-lk45q"
Jul 17 10:10:07.548033 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:07.547997    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d7247e2-690f-4fa1-858a-b6d1c2d98f43-maas-api-service-ca\") pod \"0d7247e2-690f-4fa1-858a-b6d1c2d98f43\" (UID: \"0d7247e2-690f-4fa1-858a-b6d1c2d98f43\") "
Jul 17 10:10:07.548381 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:07.548355    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d7247e2-690f-4fa1-858a-b6d1c2d98f43-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0d7247e2-690f-4fa1-858a-b6d1c2d98f43" (UID: "0d7247e2-690f-4fa1-858a-b6d1c2d98f43"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:10:07.649791 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:07.649713    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d7247e2-690f-4fa1-858a-b6d1c2d98f43-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:10:08.308596 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:08.308563    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6bff328f-lk45q"
Jul 17 10:10:08.309098 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:08.308593    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6bff328f-lk45q" event={"ID":"0d7247e2-690f-4fa1-858a-b6d1c2d98f43","Type":"ContainerDied","Data":"f7a87f39f5ab5d4c6a96d3d4aa1002752ff8d14c9be8690881c3ab72c2765ff1"}
Jul 17 10:10:08.309098 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:08.308647    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f7a87f39f5ab5d4c6a96d3d4aa1002752ff8d14c9be8690881c3ab72c2765ff1"
Jul 17 10:10:08.899586 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:08.899553    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36a00ba7-3c15-48c9-a5b6-47aec1d29bc1" path="/var/lib/kubelet/pods/36a00ba7-3c15-48c9-a5b6-47aec1d29bc1/volumes"
Jul 17 10:10:15.603117 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:15.603074    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-brrfj"]
Jul 17 10:10:15.603774 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:15.603751    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36a00ba7-3c15-48c9-a5b6-47aec1d29bc1" containerName="revoke-keys"
Jul 17 10:10:15.603830 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:15.603779    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="36a00ba7-3c15-48c9-a5b6-47aec1d29bc1" containerName="revoke-keys"
Jul 17 10:10:15.603830 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:15.603800    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36a00ba7-3c15-48c9-a5b6-47aec1d29bc1" containerName="revoke-keys"
Jul 17 10:10:15.603830 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:15.603810    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="36a00ba7-3c15-48c9-a5b6-47aec1d29bc1" containerName="revoke-keys"
Jul 17 10:10:15.603830 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:15.603827    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d7247e2-690f-4fa1-858a-b6d1c2d98f43" containerName="revoke-keys"
Jul 17 10:10:15.603956 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:15.603838    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d7247e2-690f-4fa1-858a-b6d1c2d98f43" containerName="revoke-keys"
Jul 17 10:10:15.603991 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:15.603957    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="36a00ba7-3c15-48c9-a5b6-47aec1d29bc1" containerName="revoke-keys"
Jul 17 10:10:15.603991 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:15.603972    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d7247e2-690f-4fa1-858a-b6d1c2d98f43" containerName="revoke-keys"
Jul 17 10:10:15.603991 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:15.603986    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="36a00ba7-3c15-48c9-a5b6-47aec1d29bc1" containerName="revoke-keys"
Jul 17 10:10:15.607827 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:15.607808    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-brrfj"
Jul 17 10:10:15.610790 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:15.610765    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:10:15.613066 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:15.613042    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-brrfj"]
Jul 17 10:10:15.731536 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:15.731500    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d89bac6-c784-45bf-8b03-8c78d9a0d99c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-brrfj\" (UID: \"5d89bac6-c784-45bf-8b03-8c78d9a0d99c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-brrfj"
Jul 17 10:10:15.832211 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:15.832176    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d89bac6-c784-45bf-8b03-8c78d9a0d99c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-brrfj\" (UID: \"5d89bac6-c784-45bf-8b03-8c78d9a0d99c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-brrfj"
Jul 17 10:10:15.832796 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:15.832774    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d89bac6-c784-45bf-8b03-8c78d9a0d99c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-brrfj\" (UID: \"5d89bac6-c784-45bf-8b03-8c78d9a0d99c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-brrfj"
Jul 17 10:10:15.921365 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:15.921332    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-brrfj"
Jul 17 10:10:16.074956 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:16.068057    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-brrfj"]
Jul 17 10:10:16.188869 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:16.188785    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw"]
Jul 17 10:10:16.189285 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:16.189265    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36a00ba7-3c15-48c9-a5b6-47aec1d29bc1" containerName="revoke-keys"
Jul 17 10:10:16.189340 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:16.189287    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="36a00ba7-3c15-48c9-a5b6-47aec1d29bc1" containerName="revoke-keys"
Jul 17 10:10:16.189395 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:16.189386    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="36a00ba7-3c15-48c9-a5b6-47aec1d29bc1" containerName="revoke-keys"
Jul 17 10:10:16.193831 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:16.193812    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw"
Jul 17 10:10:16.199539 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:16.199514    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw"]
Jul 17 10:10:16.235090 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:16.235055    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64c97969-665b-4dfe-8a41-72effc759feb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw\" (UID: \"64c97969-665b-4dfe-8a41-72effc759feb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw"
Jul 17 10:10:16.335757 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:16.335724    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64c97969-665b-4dfe-8a41-72effc759feb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw\" (UID: \"64c97969-665b-4dfe-8a41-72effc759feb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw"
Jul 17 10:10:16.336356 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:16.336335    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64c97969-665b-4dfe-8a41-72effc759feb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw\" (UID: \"64c97969-665b-4dfe-8a41-72effc759feb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw"
Jul 17 10:10:16.355623 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:16.355586    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-brrfj" event={"ID":"5d89bac6-c784-45bf-8b03-8c78d9a0d99c","Type":"ContainerStarted","Data":"3316b406b20b697f9510b0c8e2d3ad1a2880314629d62e93364a5027b1114b71"}
Jul 17 10:10:16.508657 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:16.508583    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw"
Jul 17 10:10:16.851944 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:16.851920    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw"]
Jul 17 10:10:16.853899 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:10:16.853870    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64c97969_665b_4dfe_8a41_72effc759feb.slice/crio-ef2b4b40853defa5aef5d2540e61f1139129c7b2c8c37bcc53016ba25fc9d7eb WatchSource:0}: Error finding container ef2b4b40853defa5aef5d2540e61f1139129c7b2c8c37bcc53016ba25fc9d7eb: Status 404 returned error can't find the container with id ef2b4b40853defa5aef5d2540e61f1139129c7b2c8c37bcc53016ba25fc9d7eb
Jul 17 10:10:17.213626 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:17.213587    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n"]
Jul 17 10:10:17.217554 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:17.217536    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n"
Jul 17 10:10:17.226093 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:17.226067    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n"]
Jul 17 10:10:17.344532 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:17.344500    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d0f53e4-6cd0-479c-99b9-dd499a374c89-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n\" (UID: \"5d0f53e4-6cd0-479c-99b9-dd499a374c89\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n"
Jul 17 10:10:17.363141 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:17.363057    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw" event={"ID":"64c97969-665b-4dfe-8a41-72effc759feb","Type":"ContainerStarted","Data":"2b492584e61c354d788e6b9f67ba390a66cebe95803f8f0a1a5d0f31c1c2900f"}
Jul 17 10:10:17.363141 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:17.363108    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw" event={"ID":"64c97969-665b-4dfe-8a41-72effc759feb","Type":"ContainerStarted","Data":"ef2b4b40853defa5aef5d2540e61f1139129c7b2c8c37bcc53016ba25fc9d7eb"}
Jul 17 10:10:17.364793 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:17.364765    2574 generic.go:356] "Generic (PLEG): container finished" podID="5d89bac6-c784-45bf-8b03-8c78d9a0d99c" containerID="58feec5cc9233276cdb16793dc419a445e7bcf7bd60f3b9ce70f68450ed1fc1a" exitCode=6
Jul 17 10:10:17.364927 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:17.364845    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-brrfj" event={"ID":"5d89bac6-c784-45bf-8b03-8c78d9a0d99c","Type":"ContainerDied","Data":"58feec5cc9233276cdb16793dc419a445e7bcf7bd60f3b9ce70f68450ed1fc1a"}
Jul 17 10:10:17.365008 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:17.364995    2574 scope.go:117] "RemoveContainer" containerID="58feec5cc9233276cdb16793dc419a445e7bcf7bd60f3b9ce70f68450ed1fc1a"
Jul 17 10:10:17.379602 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:17.379481    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw" podStartSLOduration=1.379463162 podStartE2EDuration="1.379463162s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:10:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:10:17.377566878 +0000 UTC m=+2614.960205276" watchObservedRunningTime="2026-07-17 10:10:17.379463162 +0000 UTC m=+2614.962101541"
Jul 17 10:10:17.445432 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:17.445392    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d0f53e4-6cd0-479c-99b9-dd499a374c89-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n\" (UID: \"5d0f53e4-6cd0-479c-99b9-dd499a374c89\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n"
Jul 17 10:10:17.446074 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:17.446050    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d0f53e4-6cd0-479c-99b9-dd499a374c89-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n\" (UID: \"5d0f53e4-6cd0-479c-99b9-dd499a374c89\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n"
Jul 17 10:10:17.529005 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:17.528982    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n"
Jul 17 10:10:17.678192 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:17.678166    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n"]
Jul 17 10:10:17.680842 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:10:17.680818    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d0f53e4_6cd0_479c_99b9_dd499a374c89.slice/crio-ee51379f1b834bd5d121b47000dc9e46718497b63437116f37e8e835182d396d WatchSource:0}: Error finding container ee51379f1b834bd5d121b47000dc9e46718497b63437116f37e8e835182d396d: Status 404 returned error can't find the container with id ee51379f1b834bd5d121b47000dc9e46718497b63437116f37e8e835182d396d
Jul 17 10:10:18.372846 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:18.372812    2574 generic.go:356] "Generic (PLEG): container finished" podID="64c97969-665b-4dfe-8a41-72effc759feb" containerID="2b492584e61c354d788e6b9f67ba390a66cebe95803f8f0a1a5d0f31c1c2900f" exitCode=6
Jul 17 10:10:18.373309 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:18.372891    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw" event={"ID":"64c97969-665b-4dfe-8a41-72effc759feb","Type":"ContainerDied","Data":"2b492584e61c354d788e6b9f67ba390a66cebe95803f8f0a1a5d0f31c1c2900f"}
Jul 17 10:10:18.373309 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:18.373105    2574 scope.go:117] "RemoveContainer" containerID="2b492584e61c354d788e6b9f67ba390a66cebe95803f8f0a1a5d0f31c1c2900f"
Jul 17 10:10:18.374861 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:18.374838    2574 generic.go:356] "Generic (PLEG): container finished" podID="5d89bac6-c784-45bf-8b03-8c78d9a0d99c" containerID="b2fbef6766dec6f7f1e6d1ad332bef10b17b288e6516dc67f2cc9376789ffa90" exitCode=6
Jul 17 10:10:18.374992 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:18.374920    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-brrfj" event={"ID":"5d89bac6-c784-45bf-8b03-8c78d9a0d99c","Type":"ContainerDied","Data":"b2fbef6766dec6f7f1e6d1ad332bef10b17b288e6516dc67f2cc9376789ffa90"}
Jul 17 10:10:18.374992 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:18.374984    2574 scope.go:117] "RemoveContainer" containerID="58feec5cc9233276cdb16793dc419a445e7bcf7bd60f3b9ce70f68450ed1fc1a"
Jul 17 10:10:18.375217 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:18.375197    2574 scope.go:117] "RemoveContainer" containerID="b2fbef6766dec6f7f1e6d1ad332bef10b17b288e6516dc67f2cc9376789ffa90"
Jul 17 10:10:18.375556 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:10:18.375456    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-brrfj_odh-ai-gateway-infra(5d89bac6-c784-45bf-8b03-8c78d9a0d99c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-brrfj" podUID="5d89bac6-c784-45bf-8b03-8c78d9a0d99c"
Jul 17 10:10:18.376979 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:18.376899    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n" event={"ID":"5d0f53e4-6cd0-479c-99b9-dd499a374c89","Type":"ContainerStarted","Data":"c17ef222dd9102c8a4b8b8369b3ee7beb13faf49128af80782810839ce47eb4f"}
Jul 17 10:10:18.376979 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:18.376945    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n" event={"ID":"5d0f53e4-6cd0-479c-99b9-dd499a374c89","Type":"ContainerStarted","Data":"ee51379f1b834bd5d121b47000dc9e46718497b63437116f37e8e835182d396d"}
Jul 17 10:10:18.411652 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:18.411584    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n" podStartSLOduration=1.411564282 podStartE2EDuration="1.411564282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:10:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:10:18.410383367 +0000 UTC m=+2615.993021745" watchObservedRunningTime="2026-07-17 10:10:18.411564282 +0000 UTC m=+2615.994202658"
Jul 17 10:10:19.383868 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:19.383776    2574 generic.go:356] "Generic (PLEG): container finished" podID="64c97969-665b-4dfe-8a41-72effc759feb" containerID="4fbda79c5bc1ac70c0277a23d208674e6411fa4715300c75faba47450401a996" exitCode=6
Jul 17 10:10:19.384331 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:19.383869    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw" event={"ID":"64c97969-665b-4dfe-8a41-72effc759feb","Type":"ContainerDied","Data":"4fbda79c5bc1ac70c0277a23d208674e6411fa4715300c75faba47450401a996"}
Jul 17 10:10:19.384331 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:19.383919    2574 scope.go:117] "RemoveContainer" containerID="2b492584e61c354d788e6b9f67ba390a66cebe95803f8f0a1a5d0f31c1c2900f"
Jul 17 10:10:19.384331 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:19.384119    2574 scope.go:117] "RemoveContainer" containerID="4fbda79c5bc1ac70c0277a23d208674e6411fa4715300c75faba47450401a996"
Jul 17 10:10:19.384491 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:10:19.384396    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw_odh-ai-gateway-infra(64c97969-665b-4dfe-8a41-72effc759feb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw" podUID="64c97969-665b-4dfe-8a41-72effc759feb"
Jul 17 10:10:19.385731 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:19.385714    2574 scope.go:117] "RemoveContainer" containerID="b2fbef6766dec6f7f1e6d1ad332bef10b17b288e6516dc67f2cc9376789ffa90"
Jul 17 10:10:19.385992 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:10:19.385971    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-brrfj_odh-ai-gateway-infra(5d89bac6-c784-45bf-8b03-8c78d9a0d99c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-brrfj" podUID="5d89bac6-c784-45bf-8b03-8c78d9a0d99c"
Jul 17 10:10:19.386914 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:19.386892    2574 generic.go:356] "Generic (PLEG): container finished" podID="5d0f53e4-6cd0-479c-99b9-dd499a374c89" containerID="c17ef222dd9102c8a4b8b8369b3ee7beb13faf49128af80782810839ce47eb4f" exitCode=6
Jul 17 10:10:19.386994 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:19.386940    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n" event={"ID":"5d0f53e4-6cd0-479c-99b9-dd499a374c89","Type":"ContainerDied","Data":"c17ef222dd9102c8a4b8b8369b3ee7beb13faf49128af80782810839ce47eb4f"}
Jul 17 10:10:19.387096 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:19.387081    2574 scope.go:117] "RemoveContainer" containerID="c17ef222dd9102c8a4b8b8369b3ee7beb13faf49128af80782810839ce47eb4f"
Jul 17 10:10:20.397455 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:20.397424    2574 scope.go:117] "RemoveContainer" containerID="4fbda79c5bc1ac70c0277a23d208674e6411fa4715300c75faba47450401a996"
Jul 17 10:10:20.397919 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:10:20.397652    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw_odh-ai-gateway-infra(64c97969-665b-4dfe-8a41-72effc759feb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw" podUID="64c97969-665b-4dfe-8a41-72effc759feb"
Jul 17 10:10:20.398979 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:20.398954    2574 generic.go:356] "Generic (PLEG): container finished" podID="5d0f53e4-6cd0-479c-99b9-dd499a374c89" containerID="1b5944397d573489743cf3e73c93b7171307e3269a49070826db86acceabfd02" exitCode=6
Jul 17 10:10:20.399097 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:20.398998    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n" event={"ID":"5d0f53e4-6cd0-479c-99b9-dd499a374c89","Type":"ContainerDied","Data":"1b5944397d573489743cf3e73c93b7171307e3269a49070826db86acceabfd02"}
Jul 17 10:10:20.399097 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:20.399040    2574 scope.go:117] "RemoveContainer" containerID="c17ef222dd9102c8a4b8b8369b3ee7beb13faf49128af80782810839ce47eb4f"
Jul 17 10:10:20.399188 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:20.399152    2574 scope.go:117] "RemoveContainer" containerID="1b5944397d573489743cf3e73c93b7171307e3269a49070826db86acceabfd02"
Jul 17 10:10:20.399397 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:10:20.399374    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n_odh-ai-gateway-infra(5d0f53e4-6cd0-479c-99b9-dd499a374c89)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n" podUID="5d0f53e4-6cd0-479c-99b9-dd499a374c89"
Jul 17 10:10:21.406863 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:21.406834    2574 scope.go:117] "RemoveContainer" containerID="1b5944397d573489743cf3e73c93b7171307e3269a49070826db86acceabfd02"
Jul 17 10:10:21.407362 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:10:21.407088    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n_odh-ai-gateway-infra(5d0f53e4-6cd0-479c-99b9-dd499a374c89)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n" podUID="5d0f53e4-6cd0-479c-99b9-dd499a374c89"
Jul 17 10:10:30.891980 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:30.891944    2574 scope.go:117] "RemoveContainer" containerID="b2fbef6766dec6f7f1e6d1ad332bef10b17b288e6516dc67f2cc9376789ffa90"
Jul 17 10:10:31.467915 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:31.467829    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-brrfj" event={"ID":"5d89bac6-c784-45bf-8b03-8c78d9a0d99c","Type":"ContainerStarted","Data":"5c7c0156e304ab192fac137d6e0646d51a0dba911eee6c6e94f335c635ef37d4"}
Jul 17 10:10:31.485397 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:31.485349    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-brrfj" podStartSLOduration=16.485334541 podStartE2EDuration="16.485334541s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:10:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:10:31.484856047 +0000 UTC m=+2629.067494425" watchObservedRunningTime="2026-07-17 10:10:31.485334541 +0000 UTC m=+2629.067972916"
Jul 17 10:10:31.891811 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:31.891732    2574 scope.go:117] "RemoveContainer" containerID="4fbda79c5bc1ac70c0277a23d208674e6411fa4715300c75faba47450401a996"
Jul 17 10:10:32.475286 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:32.475168    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw" event={"ID":"64c97969-665b-4dfe-8a41-72effc759feb","Type":"ContainerStarted","Data":"44f7679eadbe94fb49ec9bf8c47b906663e75a5ec101dd7d6ef07a12e5cb0d14"}
Jul 17 10:10:32.477139 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:32.477113    2574 generic.go:356] "Generic (PLEG): container finished" podID="5d89bac6-c784-45bf-8b03-8c78d9a0d99c" containerID="5c7c0156e304ab192fac137d6e0646d51a0dba911eee6c6e94f335c635ef37d4" exitCode=6
Jul 17 10:10:32.477269 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:32.477186    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-brrfj" event={"ID":"5d89bac6-c784-45bf-8b03-8c78d9a0d99c","Type":"ContainerDied","Data":"5c7c0156e304ab192fac137d6e0646d51a0dba911eee6c6e94f335c635ef37d4"}
Jul 17 10:10:32.477269 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:32.477252    2574 scope.go:117] "RemoveContainer" containerID="b2fbef6766dec6f7f1e6d1ad332bef10b17b288e6516dc67f2cc9376789ffa90"
Jul 17 10:10:32.477536 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:32.477519    2574 scope.go:117] "RemoveContainer" containerID="5c7c0156e304ab192fac137d6e0646d51a0dba911eee6c6e94f335c635ef37d4"
Jul 17 10:10:32.477745 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:10:32.477724    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-brrfj_odh-ai-gateway-infra(5d89bac6-c784-45bf-8b03-8c78d9a0d99c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-brrfj" podUID="5d89bac6-c784-45bf-8b03-8c78d9a0d99c"
Jul 17 10:10:32.497841 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:32.497814    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-brrfj"]
Jul 17 10:10:32.894892 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:32.894813    2574 scope.go:117] "RemoveContainer" containerID="1b5944397d573489743cf3e73c93b7171307e3269a49070826db86acceabfd02"
Jul 17 10:10:33.484961 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:33.484921    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n" event={"ID":"5d0f53e4-6cd0-479c-99b9-dd499a374c89","Type":"ContainerStarted","Data":"4d56098887f82b297fa682f77802ecdaeb737775e309da2bce23026653edd738"}
Jul 17 10:10:33.486875 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:33.486850    2574 generic.go:356] "Generic (PLEG): container finished" podID="64c97969-665b-4dfe-8a41-72effc759feb" containerID="44f7679eadbe94fb49ec9bf8c47b906663e75a5ec101dd7d6ef07a12e5cb0d14" exitCode=6
Jul 17 10:10:33.486988 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:33.486919    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw" event={"ID":"64c97969-665b-4dfe-8a41-72effc759feb","Type":"ContainerDied","Data":"44f7679eadbe94fb49ec9bf8c47b906663e75a5ec101dd7d6ef07a12e5cb0d14"}
Jul 17 10:10:33.487052 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:33.486988    2574 scope.go:117] "RemoveContainer" containerID="4fbda79c5bc1ac70c0277a23d208674e6411fa4715300c75faba47450401a996"
Jul 17 10:10:33.487285 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:33.487269    2574 scope.go:117] "RemoveContainer" containerID="44f7679eadbe94fb49ec9bf8c47b906663e75a5ec101dd7d6ef07a12e5cb0d14"
Jul 17 10:10:33.487506 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:10:33.487485    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw_odh-ai-gateway-infra(64c97969-665b-4dfe-8a41-72effc759feb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw" podUID="64c97969-665b-4dfe-8a41-72effc759feb"
Jul 17 10:10:33.501789 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:33.501717    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw"]
Jul 17 10:10:33.626775 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:33.626748    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-brrfj"
Jul 17 10:10:33.810524 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:33.810422    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d89bac6-c784-45bf-8b03-8c78d9a0d99c-maas-api-service-ca\") pod \"5d89bac6-c784-45bf-8b03-8c78d9a0d99c\" (UID: \"5d89bac6-c784-45bf-8b03-8c78d9a0d99c\") "
Jul 17 10:10:33.810799 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:33.810777    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d89bac6-c784-45bf-8b03-8c78d9a0d99c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5d89bac6-c784-45bf-8b03-8c78d9a0d99c" (UID: "5d89bac6-c784-45bf-8b03-8c78d9a0d99c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:10:33.911111 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:33.911065    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d89bac6-c784-45bf-8b03-8c78d9a0d99c-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:10:34.497086 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:34.497058    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-brrfj"
Jul 17 10:10:34.497086 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:34.497071    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-brrfj" event={"ID":"5d89bac6-c784-45bf-8b03-8c78d9a0d99c","Type":"ContainerDied","Data":"3316b406b20b697f9510b0c8e2d3ad1a2880314629d62e93364a5027b1114b71"}
Jul 17 10:10:34.497548 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:34.497133    2574 scope.go:117] "RemoveContainer" containerID="5c7c0156e304ab192fac137d6e0646d51a0dba911eee6c6e94f335c635ef37d4"
Jul 17 10:10:34.499384 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:34.499347    2574 generic.go:356] "Generic (PLEG): container finished" podID="5d0f53e4-6cd0-479c-99b9-dd499a374c89" containerID="4d56098887f82b297fa682f77802ecdaeb737775e309da2bce23026653edd738" exitCode=6
Jul 17 10:10:34.499502 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:34.499413    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n" event={"ID":"5d0f53e4-6cd0-479c-99b9-dd499a374c89","Type":"ContainerDied","Data":"4d56098887f82b297fa682f77802ecdaeb737775e309da2bce23026653edd738"}
Jul 17 10:10:34.499818 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:34.499747    2574 scope.go:117] "RemoveContainer" containerID="4d56098887f82b297fa682f77802ecdaeb737775e309da2bce23026653edd738"
Jul 17 10:10:34.500037 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:10:34.500012    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n_odh-ai-gateway-infra(5d0f53e4-6cd0-479c-99b9-dd499a374c89)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n" podUID="5d0f53e4-6cd0-479c-99b9-dd499a374c89"
Jul 17 10:10:34.506903 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:34.506886    2574 scope.go:117] "RemoveContainer" containerID="1b5944397d573489743cf3e73c93b7171307e3269a49070826db86acceabfd02"
Jul 17 10:10:34.512440 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:34.512416    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n"]
Jul 17 10:10:34.559643 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:34.559605    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-brrfj"]
Jul 17 10:10:34.564835 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:34.564808    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-brrfj"]
Jul 17 10:10:34.647125 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:34.647098    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw"
Jul 17 10:10:34.820614 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:34.820536    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64c97969-665b-4dfe-8a41-72effc759feb-maas-api-service-ca\") pod \"64c97969-665b-4dfe-8a41-72effc759feb\" (UID: \"64c97969-665b-4dfe-8a41-72effc759feb\") "
Jul 17 10:10:34.820929 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:34.820906    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64c97969-665b-4dfe-8a41-72effc759feb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "64c97969-665b-4dfe-8a41-72effc759feb" (UID: "64c97969-665b-4dfe-8a41-72effc759feb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:10:34.898473 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:34.898442    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d89bac6-c784-45bf-8b03-8c78d9a0d99c" path="/var/lib/kubelet/pods/5d89bac6-c784-45bf-8b03-8c78d9a0d99c/volumes"
Jul 17 10:10:34.921954 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:34.921917    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/64c97969-665b-4dfe-8a41-72effc759feb-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:10:35.506765 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:35.506734    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw"
Jul 17 10:10:35.506765 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:35.506747    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw" event={"ID":"64c97969-665b-4dfe-8a41-72effc759feb","Type":"ContainerDied","Data":"ef2b4b40853defa5aef5d2540e61f1139129c7b2c8c37bcc53016ba25fc9d7eb"}
Jul 17 10:10:35.507436 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:35.506844    2574 scope.go:117] "RemoveContainer" containerID="44f7679eadbe94fb49ec9bf8c47b906663e75a5ec101dd7d6ef07a12e5cb0d14"
Jul 17 10:10:35.525398 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:35.525362    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw"]
Jul 17 10:10:35.532999 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:35.532974    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-mllpw"]
Jul 17 10:10:35.658821 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:35.658798    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n"
Jul 17 10:10:35.832079 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:35.831992    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d0f53e4-6cd0-479c-99b9-dd499a374c89-maas-api-service-ca\") pod \"5d0f53e4-6cd0-479c-99b9-dd499a374c89\" (UID: \"5d0f53e4-6cd0-479c-99b9-dd499a374c89\") "
Jul 17 10:10:35.832355 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:35.832335    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d0f53e4-6cd0-479c-99b9-dd499a374c89-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5d0f53e4-6cd0-479c-99b9-dd499a374c89" (UID: "5d0f53e4-6cd0-479c-99b9-dd499a374c89"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:10:35.932696 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:35.932663    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d0f53e4-6cd0-479c-99b9-dd499a374c89-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:10:36.517447 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:36.517411    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n"
Jul 17 10:10:36.517944 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:36.517410    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n" event={"ID":"5d0f53e4-6cd0-479c-99b9-dd499a374c89","Type":"ContainerDied","Data":"ee51379f1b834bd5d121b47000dc9e46718497b63437116f37e8e835182d396d"}
Jul 17 10:10:36.517944 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:36.517574    2574 scope.go:117] "RemoveContainer" containerID="4d56098887f82b297fa682f77802ecdaeb737775e309da2bce23026653edd738"
Jul 17 10:10:36.538961 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:36.538933    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n"]
Jul 17 10:10:36.544908 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:36.544879    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-jpl4n"]
Jul 17 10:10:36.898234 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:36.898154    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d0f53e4-6cd0-479c-99b9-dd499a374c89" path="/var/lib/kubelet/pods/5d0f53e4-6cd0-479c-99b9-dd499a374c89/volumes"
Jul 17 10:10:36.898526 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:36.898512    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64c97969-665b-4dfe-8a41-72effc759feb" path="/var/lib/kubelet/pods/64c97969-665b-4dfe-8a41-72effc759feb/volumes"
Jul 17 10:10:38.348050 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.348018    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-847pn"]
Jul 17 10:10:38.348471 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.348459    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d0f53e4-6cd0-479c-99b9-dd499a374c89" containerName="revoke-keys"
Jul 17 10:10:38.348514 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.348470    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d0f53e4-6cd0-479c-99b9-dd499a374c89" containerName="revoke-keys"
Jul 17 10:10:38.348514 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.348481    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d0f53e4-6cd0-479c-99b9-dd499a374c89" containerName="revoke-keys"
Jul 17 10:10:38.348514 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.348486    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d0f53e4-6cd0-479c-99b9-dd499a374c89" containerName="revoke-keys"
Jul 17 10:10:38.348514 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.348497    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d89bac6-c784-45bf-8b03-8c78d9a0d99c" containerName="revoke-keys"
Jul 17 10:10:38.348514 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.348503    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d89bac6-c784-45bf-8b03-8c78d9a0d99c" containerName="revoke-keys"
Jul 17 10:10:38.348514 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.348513    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d89bac6-c784-45bf-8b03-8c78d9a0d99c" containerName="revoke-keys"
Jul 17 10:10:38.348702 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.348518    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d89bac6-c784-45bf-8b03-8c78d9a0d99c" containerName="revoke-keys"
Jul 17 10:10:38.348702 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.348530    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64c97969-665b-4dfe-8a41-72effc759feb" containerName="revoke-keys"
Jul 17 10:10:38.348702 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.348535    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="64c97969-665b-4dfe-8a41-72effc759feb" containerName="revoke-keys"
Jul 17 10:10:38.348702 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.348544    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64c97969-665b-4dfe-8a41-72effc759feb" containerName="revoke-keys"
Jul 17 10:10:38.348702 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.348549    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="64c97969-665b-4dfe-8a41-72effc759feb" containerName="revoke-keys"
Jul 17 10:10:38.348702 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.348555    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64c97969-665b-4dfe-8a41-72effc759feb" containerName="revoke-keys"
Jul 17 10:10:38.348702 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.348559    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="64c97969-665b-4dfe-8a41-72effc759feb" containerName="revoke-keys"
Jul 17 10:10:38.348702 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.348617    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d89bac6-c784-45bf-8b03-8c78d9a0d99c" containerName="revoke-keys"
Jul 17 10:10:38.348702 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.348624    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d0f53e4-6cd0-479c-99b9-dd499a374c89" containerName="revoke-keys"
Jul 17 10:10:38.348702 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.348630    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="64c97969-665b-4dfe-8a41-72effc759feb" containerName="revoke-keys"
Jul 17 10:10:38.348702 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.348639    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="64c97969-665b-4dfe-8a41-72effc759feb" containerName="revoke-keys"
Jul 17 10:10:38.348702 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.348645    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d89bac6-c784-45bf-8b03-8c78d9a0d99c" containerName="revoke-keys"
Jul 17 10:10:38.348702 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.348650    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d0f53e4-6cd0-479c-99b9-dd499a374c89" containerName="revoke-keys"
Jul 17 10:10:38.353319 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.353300    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-847pn"
Jul 17 10:10:38.356453 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.356422    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:10:38.356661 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.356641    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/511364ca-1fe6-4302-b9d2-22f355f0c50d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-847pn\" (UID: \"511364ca-1fe6-4302-b9d2-22f355f0c50d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-847pn"
Jul 17 10:10:38.362008 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.361979    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-847pn"]
Jul 17 10:10:38.457612 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.457582    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/511364ca-1fe6-4302-b9d2-22f355f0c50d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-847pn\" (UID: \"511364ca-1fe6-4302-b9d2-22f355f0c50d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-847pn"
Jul 17 10:10:38.458275 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.458220    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/511364ca-1fe6-4302-b9d2-22f355f0c50d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-847pn\" (UID: \"511364ca-1fe6-4302-b9d2-22f355f0c50d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-847pn"
Jul 17 10:10:38.664546 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.664521    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-847pn"
Jul 17 10:10:38.807275 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:10:38.807205    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod511364ca_1fe6_4302_b9d2_22f355f0c50d.slice/crio-0c5185d72e66ee3edebb52360ac04519778e5f35988cefe7b332b7b602faae0f WatchSource:0}: Error finding container 0c5185d72e66ee3edebb52360ac04519778e5f35988cefe7b332b7b602faae0f: Status 404 returned error can't find the container with id 0c5185d72e66ee3edebb52360ac04519778e5f35988cefe7b332b7b602faae0f
Jul 17 10:10:38.807413 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:38.807281    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-847pn"]
Jul 17 10:10:39.539163 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:39.539122    2574 generic.go:356] "Generic (PLEG): container finished" podID="511364ca-1fe6-4302-b9d2-22f355f0c50d" containerID="ad693f8503cc7605787c57714ec6b8a73ac04e1b4d368659679b4633c481b5f5" exitCode=6
Jul 17 10:10:39.539645 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:39.539214    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-847pn" event={"ID":"511364ca-1fe6-4302-b9d2-22f355f0c50d","Type":"ContainerDied","Data":"ad693f8503cc7605787c57714ec6b8a73ac04e1b4d368659679b4633c481b5f5"}
Jul 17 10:10:39.539645 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:39.539288    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-847pn" event={"ID":"511364ca-1fe6-4302-b9d2-22f355f0c50d","Type":"ContainerStarted","Data":"0c5185d72e66ee3edebb52360ac04519778e5f35988cefe7b332b7b602faae0f"}
Jul 17 10:10:39.539645 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:39.539468    2574 scope.go:117] "RemoveContainer" containerID="ad693f8503cc7605787c57714ec6b8a73ac04e1b4d368659679b4633c481b5f5"
Jul 17 10:10:40.546763 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:40.546727    2574 generic.go:356] "Generic (PLEG): container finished" podID="511364ca-1fe6-4302-b9d2-22f355f0c50d" containerID="dc49b14138198c501476a7a402095a5a877399469c0b27c339a309e0fd859e24" exitCode=6
Jul 17 10:10:40.547291 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:40.546821    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-847pn" event={"ID":"511364ca-1fe6-4302-b9d2-22f355f0c50d","Type":"ContainerDied","Data":"dc49b14138198c501476a7a402095a5a877399469c0b27c339a309e0fd859e24"}
Jul 17 10:10:40.547291 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:40.546883    2574 scope.go:117] "RemoveContainer" containerID="ad693f8503cc7605787c57714ec6b8a73ac04e1b4d368659679b4633c481b5f5"
Jul 17 10:10:40.547291 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:40.547084    2574 scope.go:117] "RemoveContainer" containerID="dc49b14138198c501476a7a402095a5a877399469c0b27c339a309e0fd859e24"
Jul 17 10:10:40.547468 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:10:40.547405    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-847pn_odh-ai-gateway-infra(511364ca-1fe6-4302-b9d2-22f355f0c50d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-847pn" podUID="511364ca-1fe6-4302-b9d2-22f355f0c50d"
Jul 17 10:10:41.554711 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:41.554681    2574 scope.go:117] "RemoveContainer" containerID="dc49b14138198c501476a7a402095a5a877399469c0b27c339a309e0fd859e24"
Jul 17 10:10:41.555087 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:10:41.554876    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-847pn_odh-ai-gateway-infra(511364ca-1fe6-4302-b9d2-22f355f0c50d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-847pn" podUID="511364ca-1fe6-4302-b9d2-22f355f0c50d"
Jul 17 10:10:52.894680 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:52.894637    2574 scope.go:117] "RemoveContainer" containerID="dc49b14138198c501476a7a402095a5a877399469c0b27c339a309e0fd859e24"
Jul 17 10:10:53.620449 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:53.620417    2574 generic.go:356] "Generic (PLEG): container finished" podID="511364ca-1fe6-4302-b9d2-22f355f0c50d" containerID="fec87cc699e8f97d9a59a58b7f9a82d69f3a6cd53654f213a04dfeb59ffb4cbe" exitCode=6
Jul 17 10:10:53.620611 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:53.620488    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-847pn" event={"ID":"511364ca-1fe6-4302-b9d2-22f355f0c50d","Type":"ContainerDied","Data":"fec87cc699e8f97d9a59a58b7f9a82d69f3a6cd53654f213a04dfeb59ffb4cbe"}
Jul 17 10:10:53.620611 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:53.620549    2574 scope.go:117] "RemoveContainer" containerID="dc49b14138198c501476a7a402095a5a877399469c0b27c339a309e0fd859e24"
Jul 17 10:10:53.620869 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:53.620852    2574 scope.go:117] "RemoveContainer" containerID="fec87cc699e8f97d9a59a58b7f9a82d69f3a6cd53654f213a04dfeb59ffb4cbe"
Jul 17 10:10:53.621152 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:10:53.621133    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-847pn_odh-ai-gateway-infra(511364ca-1fe6-4302-b9d2-22f355f0c50d)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-847pn" podUID="511364ca-1fe6-4302-b9d2-22f355f0c50d"
Jul 17 10:10:54.651012 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:54.650978    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-847pn"]
Jul 17 10:10:54.799160 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:54.799135    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-847pn"
Jul 17 10:10:54.920578 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:54.920533    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/511364ca-1fe6-4302-b9d2-22f355f0c50d-maas-api-service-ca\") pod \"511364ca-1fe6-4302-b9d2-22f355f0c50d\" (UID: \"511364ca-1fe6-4302-b9d2-22f355f0c50d\") "
Jul 17 10:10:54.920865 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:54.920843    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/511364ca-1fe6-4302-b9d2-22f355f0c50d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "511364ca-1fe6-4302-b9d2-22f355f0c50d" (UID: "511364ca-1fe6-4302-b9d2-22f355f0c50d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:10:55.021653 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:55.021623    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/511364ca-1fe6-4302-b9d2-22f355f0c50d-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:10:55.633952 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:55.633917    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-847pn"
Jul 17 10:10:55.634129 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:55.633951    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-847pn" event={"ID":"511364ca-1fe6-4302-b9d2-22f355f0c50d","Type":"ContainerDied","Data":"0c5185d72e66ee3edebb52360ac04519778e5f35988cefe7b332b7b602faae0f"}
Jul 17 10:10:55.634129 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:55.634016    2574 scope.go:117] "RemoveContainer" containerID="fec87cc699e8f97d9a59a58b7f9a82d69f3a6cd53654f213a04dfeb59ffb4cbe"
Jul 17 10:10:55.655706 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:55.655679    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-847pn"]
Jul 17 10:10:55.662120 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:55.662101    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-847pn"]
Jul 17 10:10:56.898300 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:10:56.898268    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="511364ca-1fe6-4302-b9d2-22f355f0c50d" path="/var/lib/kubelet/pods/511364ca-1fe6-4302-b9d2-22f355f0c50d/volumes"
Jul 17 10:11:05.642149 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:05.642107    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vxhpp"]
Jul 17 10:11:05.642758 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:05.642737    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d89bac6-c784-45bf-8b03-8c78d9a0d99c" containerName="revoke-keys"
Jul 17 10:11:05.642847 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:05.642762    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d89bac6-c784-45bf-8b03-8c78d9a0d99c" containerName="revoke-keys"
Jul 17 10:11:05.642847 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:05.642781    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="511364ca-1fe6-4302-b9d2-22f355f0c50d" containerName="revoke-keys"
Jul 17 10:11:05.642847 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:05.642790    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="511364ca-1fe6-4302-b9d2-22f355f0c50d" containerName="revoke-keys"
Jul 17 10:11:05.642847 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:05.642831    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d0f53e4-6cd0-479c-99b9-dd499a374c89" containerName="revoke-keys"
Jul 17 10:11:05.642847 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:05.642840    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d0f53e4-6cd0-479c-99b9-dd499a374c89" containerName="revoke-keys"
Jul 17 10:11:05.643093 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:05.642860    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="511364ca-1fe6-4302-b9d2-22f355f0c50d" containerName="revoke-keys"
Jul 17 10:11:05.643093 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:05.642869    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="511364ca-1fe6-4302-b9d2-22f355f0c50d" containerName="revoke-keys"
Jul 17 10:11:05.643093 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:05.642963    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d89bac6-c784-45bf-8b03-8c78d9a0d99c" containerName="revoke-keys"
Jul 17 10:11:05.643093 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:05.642976    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d0f53e4-6cd0-479c-99b9-dd499a374c89" containerName="revoke-keys"
Jul 17 10:11:05.643093 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:05.642987    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="64c97969-665b-4dfe-8a41-72effc759feb" containerName="revoke-keys"
Jul 17 10:11:05.643093 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:05.642999    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="511364ca-1fe6-4302-b9d2-22f355f0c50d" containerName="revoke-keys"
Jul 17 10:11:05.643093 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:05.643008    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="511364ca-1fe6-4302-b9d2-22f355f0c50d" containerName="revoke-keys"
Jul 17 10:11:05.643093 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:05.643017    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="511364ca-1fe6-4302-b9d2-22f355f0c50d" containerName="revoke-keys"
Jul 17 10:11:05.647462 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:05.647441    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vxhpp"
Jul 17 10:11:05.650413 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:05.650388    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:11:05.654158 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:05.654133    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vxhpp"]
Jul 17 10:11:05.825196 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:05.825152    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d1a1089-b418-49f5-beca-0b0f368f67ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-vxhpp\" (UID: \"2d1a1089-b418-49f5-beca-0b0f368f67ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vxhpp"
Jul 17 10:11:05.926233 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:05.926199    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d1a1089-b418-49f5-beca-0b0f368f67ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-vxhpp\" (UID: \"2d1a1089-b418-49f5-beca-0b0f368f67ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vxhpp"
Jul 17 10:11:05.926854 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:05.926835    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d1a1089-b418-49f5-beca-0b0f368f67ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-vxhpp\" (UID: \"2d1a1089-b418-49f5-beca-0b0f368f67ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vxhpp"
Jul 17 10:11:05.960187 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:05.960153    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vxhpp"
Jul 17 10:11:06.129624 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:06.129595    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vxhpp"]
Jul 17 10:11:06.132009 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:11:06.131973    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d1a1089_b418_49f5_beca_0b0f368f67ad.slice/crio-45138d8e50440957826fc1839f70801b68107bb8fabf3e0c50960f14f9ddb77f WatchSource:0}: Error finding container 45138d8e50440957826fc1839f70801b68107bb8fabf3e0c50960f14f9ddb77f: Status 404 returned error can't find the container with id 45138d8e50440957826fc1839f70801b68107bb8fabf3e0c50960f14f9ddb77f
Jul 17 10:11:06.240088 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:06.240055    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v4zml"]
Jul 17 10:11:06.240526 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:06.240513    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="511364ca-1fe6-4302-b9d2-22f355f0c50d" containerName="revoke-keys"
Jul 17 10:11:06.240575 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:06.240529    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="511364ca-1fe6-4302-b9d2-22f355f0c50d" containerName="revoke-keys"
Jul 17 10:11:06.244938 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:06.244922    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v4zml"
Jul 17 10:11:06.251921 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:06.251897    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v4zml"]
Jul 17 10:11:06.331365 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:06.331327    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac90c7ba-f6ce-4609-ad17-485048a970cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-v4zml\" (UID: \"ac90c7ba-f6ce-4609-ad17-485048a970cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v4zml"
Jul 17 10:11:06.432844 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:06.432786    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac90c7ba-f6ce-4609-ad17-485048a970cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-v4zml\" (UID: \"ac90c7ba-f6ce-4609-ad17-485048a970cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v4zml"
Jul 17 10:11:06.433376 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:06.433360    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac90c7ba-f6ce-4609-ad17-485048a970cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-v4zml\" (UID: \"ac90c7ba-f6ce-4609-ad17-485048a970cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v4zml"
Jul 17 10:11:06.557065 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:06.557039    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v4zml"
Jul 17 10:11:06.696836 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:06.696781    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vxhpp" event={"ID":"2d1a1089-b418-49f5-beca-0b0f368f67ad","Type":"ContainerStarted","Data":"d12ecdc86745949711a838c53884b3c4578e4fe36a2365c89e733def8f541ab6"}
Jul 17 10:11:06.697200 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:06.696853    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vxhpp" event={"ID":"2d1a1089-b418-49f5-beca-0b0f368f67ad","Type":"ContainerStarted","Data":"45138d8e50440957826fc1839f70801b68107bb8fabf3e0c50960f14f9ddb77f"}
Jul 17 10:11:06.697200 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:06.696889    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v4zml"]
Jul 17 10:11:06.697611 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:11:06.697585    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac90c7ba_f6ce_4609_ad17_485048a970cb.slice/crio-ee79e1561a5df79596014edae23a279e38e9db084862ea2f5baa57b575bfa61f WatchSource:0}: Error finding container ee79e1561a5df79596014edae23a279e38e9db084862ea2f5baa57b575bfa61f: Status 404 returned error can't find the container with id ee79e1561a5df79596014edae23a279e38e9db084862ea2f5baa57b575bfa61f
Jul 17 10:11:06.713864 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:06.713809    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vxhpp" podStartSLOduration=1.713792691 podStartE2EDuration="1.713792691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:11:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:11:06.711578594 +0000 UTC m=+2664.294216970" watchObservedRunningTime="2026-07-17 10:11:06.713792691 +0000 UTC m=+2664.296431066"
Jul 17 10:11:07.267687 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:07.267602    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-f7gpk"]
Jul 17 10:11:07.271270 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:07.271231    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-f7gpk"
Jul 17 10:11:07.279437 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:07.279409    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-f7gpk"]
Jul 17 10:11:07.442786 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:07.442750    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b75aebb9-ae40-4709-a131-1bd2a60be619-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-f7gpk\" (UID: \"b75aebb9-ae40-4709-a131-1bd2a60be619\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-f7gpk"
Jul 17 10:11:07.543865 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:07.543774    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b75aebb9-ae40-4709-a131-1bd2a60be619-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-f7gpk\" (UID: \"b75aebb9-ae40-4709-a131-1bd2a60be619\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-f7gpk"
Jul 17 10:11:07.544397 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:07.544377    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b75aebb9-ae40-4709-a131-1bd2a60be619-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-f7gpk\" (UID: \"b75aebb9-ae40-4709-a131-1bd2a60be619\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-f7gpk"
Jul 17 10:11:07.582996 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:07.582973    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-f7gpk"
Jul 17 10:11:07.706512 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:07.706481    2574 generic.go:356] "Generic (PLEG): container finished" podID="2d1a1089-b418-49f5-beca-0b0f368f67ad" containerID="d12ecdc86745949711a838c53884b3c4578e4fe36a2365c89e733def8f541ab6" exitCode=6
Jul 17 10:11:07.706959 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:07.706537    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vxhpp" event={"ID":"2d1a1089-b418-49f5-beca-0b0f368f67ad","Type":"ContainerDied","Data":"d12ecdc86745949711a838c53884b3c4578e4fe36a2365c89e733def8f541ab6"}
Jul 17 10:11:07.706959 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:07.706908    2574 scope.go:117] "RemoveContainer" containerID="d12ecdc86745949711a838c53884b3c4578e4fe36a2365c89e733def8f541ab6"
Jul 17 10:11:07.708676 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:07.708654    2574 generic.go:356] "Generic (PLEG): container finished" podID="ac90c7ba-f6ce-4609-ad17-485048a970cb" containerID="547a4e6ce55f9c75472c932ba3171ad7210cf67c0806e3e7ac7b09bf116ae50c" exitCode=6
Jul 17 10:11:07.708787 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:07.708754    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v4zml" event={"ID":"ac90c7ba-f6ce-4609-ad17-485048a970cb","Type":"ContainerDied","Data":"547a4e6ce55f9c75472c932ba3171ad7210cf67c0806e3e7ac7b09bf116ae50c"}
Jul 17 10:11:07.708832 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:07.708785    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v4zml" event={"ID":"ac90c7ba-f6ce-4609-ad17-485048a970cb","Type":"ContainerStarted","Data":"ee79e1561a5df79596014edae23a279e38e9db084862ea2f5baa57b575bfa61f"}
Jul 17 10:11:07.709022 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:07.709009    2574 scope.go:117] "RemoveContainer" containerID="547a4e6ce55f9c75472c932ba3171ad7210cf67c0806e3e7ac7b09bf116ae50c"
Jul 17 10:11:07.741466 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:07.741365    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-f7gpk"]
Jul 17 10:11:08.716037 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:08.716000    2574 generic.go:356] "Generic (PLEG): container finished" podID="b75aebb9-ae40-4709-a131-1bd2a60be619" containerID="2a54a978526a755c3cd687a5934f9f0bed06b400577434de12bd539b43977258" exitCode=6
Jul 17 10:11:08.716483 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:08.716087    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-f7gpk" event={"ID":"b75aebb9-ae40-4709-a131-1bd2a60be619","Type":"ContainerDied","Data":"2a54a978526a755c3cd687a5934f9f0bed06b400577434de12bd539b43977258"}
Jul 17 10:11:08.716483 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:08.716149    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-f7gpk" event={"ID":"b75aebb9-ae40-4709-a131-1bd2a60be619","Type":"ContainerStarted","Data":"aa337a0b475f200a232dca5d5346f97e0d70547b2b19d428fb04b922f29e9a60"}
Jul 17 10:11:08.716483 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:08.716327    2574 scope.go:117] "RemoveContainer" containerID="2a54a978526a755c3cd687a5934f9f0bed06b400577434de12bd539b43977258"
Jul 17 10:11:08.718338 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:08.718315    2574 generic.go:356] "Generic (PLEG): container finished" podID="2d1a1089-b418-49f5-beca-0b0f368f67ad" containerID="f8d7b68cf9b02ea890c78c5f8316d3e0d9a8c06977d19b21fa9600001a4ea9f7" exitCode=6
Jul 17 10:11:08.718452 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:08.718383    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vxhpp" event={"ID":"2d1a1089-b418-49f5-beca-0b0f368f67ad","Type":"ContainerDied","Data":"f8d7b68cf9b02ea890c78c5f8316d3e0d9a8c06977d19b21fa9600001a4ea9f7"}
Jul 17 10:11:08.718452 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:08.718440    2574 scope.go:117] "RemoveContainer" containerID="d12ecdc86745949711a838c53884b3c4578e4fe36a2365c89e733def8f541ab6"
Jul 17 10:11:08.718618 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:08.718605    2574 scope.go:117] "RemoveContainer" containerID="f8d7b68cf9b02ea890c78c5f8316d3e0d9a8c06977d19b21fa9600001a4ea9f7"
Jul 17 10:11:08.718882 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:11:08.718840    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-vxhpp_odh-ai-gateway-infra(2d1a1089-b418-49f5-beca-0b0f368f67ad)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vxhpp" podUID="2d1a1089-b418-49f5-beca-0b0f368f67ad"
Jul 17 10:11:08.720653 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:08.720633    2574 generic.go:356] "Generic (PLEG): container finished" podID="ac90c7ba-f6ce-4609-ad17-485048a970cb" containerID="2288cf500ba17a36127ad0366fb38c7638f0ebda9cd5327cf41be5d1b357270f" exitCode=6
Jul 17 10:11:08.720737 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:08.720708    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v4zml" event={"ID":"ac90c7ba-f6ce-4609-ad17-485048a970cb","Type":"ContainerDied","Data":"2288cf500ba17a36127ad0366fb38c7638f0ebda9cd5327cf41be5d1b357270f"}
Jul 17 10:11:08.720868 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:08.720853    2574 scope.go:117] "RemoveContainer" containerID="2288cf500ba17a36127ad0366fb38c7638f0ebda9cd5327cf41be5d1b357270f"
Jul 17 10:11:08.721040 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:11:08.721020    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-v4zml_odh-ai-gateway-infra(ac90c7ba-f6ce-4609-ad17-485048a970cb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v4zml" podUID="ac90c7ba-f6ce-4609-ad17-485048a970cb"
Jul 17 10:11:08.730503 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:08.730485    2574 scope.go:117] "RemoveContainer" containerID="547a4e6ce55f9c75472c932ba3171ad7210cf67c0806e3e7ac7b09bf116ae50c"
Jul 17 10:11:09.729518 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:09.729483    2574 generic.go:356] "Generic (PLEG): container finished" podID="b75aebb9-ae40-4709-a131-1bd2a60be619" containerID="91e534b386d98b8141f38e62368b8be09a834892b810d347aa3d662871efe7ad" exitCode=6
Jul 17 10:11:09.729948 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:09.729524    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-f7gpk" event={"ID":"b75aebb9-ae40-4709-a131-1bd2a60be619","Type":"ContainerDied","Data":"91e534b386d98b8141f38e62368b8be09a834892b810d347aa3d662871efe7ad"}
Jul 17 10:11:09.729948 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:09.729592    2574 scope.go:117] "RemoveContainer" containerID="2a54a978526a755c3cd687a5934f9f0bed06b400577434de12bd539b43977258"
Jul 17 10:11:09.729948 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:09.729780    2574 scope.go:117] "RemoveContainer" containerID="91e534b386d98b8141f38e62368b8be09a834892b810d347aa3d662871efe7ad"
Jul 17 10:11:09.730098 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:11:09.730063    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-f7gpk_odh-ai-gateway-infra(b75aebb9-ae40-4709-a131-1bd2a60be619)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-f7gpk" podUID="b75aebb9-ae40-4709-a131-1bd2a60be619"
Jul 17 10:11:09.731626 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:09.731608    2574 scope.go:117] "RemoveContainer" containerID="f8d7b68cf9b02ea890c78c5f8316d3e0d9a8c06977d19b21fa9600001a4ea9f7"
Jul 17 10:11:09.731829 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:11:09.731810    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-vxhpp_odh-ai-gateway-infra(2d1a1089-b418-49f5-beca-0b0f368f67ad)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vxhpp" podUID="2d1a1089-b418-49f5-beca-0b0f368f67ad"
Jul 17 10:11:09.733470 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:09.733452    2574 scope.go:117] "RemoveContainer" containerID="2288cf500ba17a36127ad0366fb38c7638f0ebda9cd5327cf41be5d1b357270f"
Jul 17 10:11:09.733670 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:11:09.733645    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-v4zml_odh-ai-gateway-infra(ac90c7ba-f6ce-4609-ad17-485048a970cb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v4zml" podUID="ac90c7ba-f6ce-4609-ad17-485048a970cb"
Jul 17 10:11:10.741152 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:10.741117    2574 scope.go:117] "RemoveContainer" containerID="91e534b386d98b8141f38e62368b8be09a834892b810d347aa3d662871efe7ad"
Jul 17 10:11:10.741615 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:11:10.741401    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-f7gpk_odh-ai-gateway-infra(b75aebb9-ae40-4709-a131-1bd2a60be619)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-f7gpk" podUID="b75aebb9-ae40-4709-a131-1bd2a60be619"
Jul 17 10:11:20.891766 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:20.891731    2574 scope.go:117] "RemoveContainer" containerID="2288cf500ba17a36127ad0366fb38c7638f0ebda9cd5327cf41be5d1b357270f"
Jul 17 10:11:20.892224 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:20.891852    2574 scope.go:117] "RemoveContainer" containerID="f8d7b68cf9b02ea890c78c5f8316d3e0d9a8c06977d19b21fa9600001a4ea9f7"
Jul 17 10:11:21.805402 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:21.805328    2574 generic.go:356] "Generic (PLEG): container finished" podID="2d1a1089-b418-49f5-beca-0b0f368f67ad" containerID="c3eed7432fcecce18d3922c95b8769eab0cd1cbd354873378ebba3eb314bedd3" exitCode=6
Jul 17 10:11:21.805557 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:21.805401    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vxhpp" event={"ID":"2d1a1089-b418-49f5-beca-0b0f368f67ad","Type":"ContainerDied","Data":"c3eed7432fcecce18d3922c95b8769eab0cd1cbd354873378ebba3eb314bedd3"}
Jul 17 10:11:21.805557 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:21.805467    2574 scope.go:117] "RemoveContainer" containerID="f8d7b68cf9b02ea890c78c5f8316d3e0d9a8c06977d19b21fa9600001a4ea9f7"
Jul 17 10:11:21.805854 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:21.805835    2574 scope.go:117] "RemoveContainer" containerID="c3eed7432fcecce18d3922c95b8769eab0cd1cbd354873378ebba3eb314bedd3"
Jul 17 10:11:21.806127 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:11:21.806102    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-vxhpp_odh-ai-gateway-infra(2d1a1089-b418-49f5-beca-0b0f368f67ad)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vxhpp" podUID="2d1a1089-b418-49f5-beca-0b0f368f67ad"
Jul 17 10:11:21.807444 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:21.807422    2574 generic.go:356] "Generic (PLEG): container finished" podID="ac90c7ba-f6ce-4609-ad17-485048a970cb" containerID="669d7a005924de9cd73437d5a38c5c04dc62c8563f5fb74ad7ce7ead6ce73b28" exitCode=6
Jul 17 10:11:21.807549 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:21.807498    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v4zml" event={"ID":"ac90c7ba-f6ce-4609-ad17-485048a970cb","Type":"ContainerDied","Data":"669d7a005924de9cd73437d5a38c5c04dc62c8563f5fb74ad7ce7ead6ce73b28"}
Jul 17 10:11:21.807806 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:21.807792    2574 scope.go:117] "RemoveContainer" containerID="669d7a005924de9cd73437d5a38c5c04dc62c8563f5fb74ad7ce7ead6ce73b28"
Jul 17 10:11:21.807992 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:11:21.807970    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-v4zml_odh-ai-gateway-infra(ac90c7ba-f6ce-4609-ad17-485048a970cb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v4zml" podUID="ac90c7ba-f6ce-4609-ad17-485048a970cb"
Jul 17 10:11:21.818758 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:21.818738    2574 scope.go:117] "RemoveContainer" containerID="2288cf500ba17a36127ad0366fb38c7638f0ebda9cd5327cf41be5d1b357270f"
Jul 17 10:11:22.836451 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:22.836417    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vxhpp"]
Jul 17 10:11:22.854940 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:22.854913    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v4zml"]
Jul 17 10:11:23.000119 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:23.000094    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vxhpp"
Jul 17 10:11:23.032273 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:23.032204    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v4zml"
Jul 17 10:11:23.092440 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:23.092353    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac90c7ba-f6ce-4609-ad17-485048a970cb-maas-api-service-ca\") pod \"ac90c7ba-f6ce-4609-ad17-485048a970cb\" (UID: \"ac90c7ba-f6ce-4609-ad17-485048a970cb\") "
Jul 17 10:11:23.092440 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:23.092420    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d1a1089-b418-49f5-beca-0b0f368f67ad-maas-api-service-ca\") pod \"2d1a1089-b418-49f5-beca-0b0f368f67ad\" (UID: \"2d1a1089-b418-49f5-beca-0b0f368f67ad\") "
Jul 17 10:11:23.092785 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:23.092761    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac90c7ba-f6ce-4609-ad17-485048a970cb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ac90c7ba-f6ce-4609-ad17-485048a970cb" (UID: "ac90c7ba-f6ce-4609-ad17-485048a970cb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:11:23.092831 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:23.092762    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d1a1089-b418-49f5-beca-0b0f368f67ad-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2d1a1089-b418-49f5-beca-0b0f368f67ad" (UID: "2d1a1089-b418-49f5-beca-0b0f368f67ad"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:11:23.193163 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:23.193135    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac90c7ba-f6ce-4609-ad17-485048a970cb-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:11:23.193163 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:23.193160    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d1a1089-b418-49f5-beca-0b0f368f67ad-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:11:23.823811 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:23.823776    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vxhpp"
Jul 17 10:11:23.823988 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:23.823778    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vxhpp" event={"ID":"2d1a1089-b418-49f5-beca-0b0f368f67ad","Type":"ContainerDied","Data":"45138d8e50440957826fc1839f70801b68107bb8fabf3e0c50960f14f9ddb77f"}
Jul 17 10:11:23.823988 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:23.823948    2574 scope.go:117] "RemoveContainer" containerID="c3eed7432fcecce18d3922c95b8769eab0cd1cbd354873378ebba3eb314bedd3"
Jul 17 10:11:23.825281 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:23.825259    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v4zml"
Jul 17 10:11:23.825402 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:23.825380    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v4zml" event={"ID":"ac90c7ba-f6ce-4609-ad17-485048a970cb","Type":"ContainerDied","Data":"ee79e1561a5df79596014edae23a279e38e9db084862ea2f5baa57b575bfa61f"}
Jul 17 10:11:23.834896 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:23.834880    2574 scope.go:117] "RemoveContainer" containerID="669d7a005924de9cd73437d5a38c5c04dc62c8563f5fb74ad7ce7ead6ce73b28"
Jul 17 10:11:23.850723 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:23.850699    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vxhpp"]
Jul 17 10:11:23.856892 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:23.856872    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vxhpp"]
Jul 17 10:11:23.871510 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:23.871488    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v4zml"]
Jul 17 10:11:23.873487 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:23.873468    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-v4zml"]
Jul 17 10:11:23.891482 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:23.891464    2574 scope.go:117] "RemoveContainer" containerID="91e534b386d98b8141f38e62368b8be09a834892b810d347aa3d662871efe7ad"
Jul 17 10:11:24.833593 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:24.833502    2574 generic.go:356] "Generic (PLEG): container finished" podID="b75aebb9-ae40-4709-a131-1bd2a60be619" containerID="1f9139ad2944836c463cf9700a18c4853c334fdd50103556d48cb96aa35f23e9" exitCode=6
Jul 17 10:11:24.833593 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:24.833576    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-f7gpk" event={"ID":"b75aebb9-ae40-4709-a131-1bd2a60be619","Type":"ContainerDied","Data":"1f9139ad2944836c463cf9700a18c4853c334fdd50103556d48cb96aa35f23e9"}
Jul 17 10:11:24.833840 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:24.833631    2574 scope.go:117] "RemoveContainer" containerID="91e534b386d98b8141f38e62368b8be09a834892b810d347aa3d662871efe7ad"
Jul 17 10:11:24.834016 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:24.833997    2574 scope.go:117] "RemoveContainer" containerID="1f9139ad2944836c463cf9700a18c4853c334fdd50103556d48cb96aa35f23e9"
Jul 17 10:11:24.834299 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:11:24.834232    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-f7gpk_odh-ai-gateway-infra(b75aebb9-ae40-4709-a131-1bd2a60be619)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-f7gpk" podUID="b75aebb9-ae40-4709-a131-1bd2a60be619"
Jul 17 10:11:24.897917 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:24.897889    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d1a1089-b418-49f5-beca-0b0f368f67ad" path="/var/lib/kubelet/pods/2d1a1089-b418-49f5-beca-0b0f368f67ad/volumes"
Jul 17 10:11:24.898289 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:24.898274    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac90c7ba-f6ce-4609-ad17-485048a970cb" path="/var/lib/kubelet/pods/ac90c7ba-f6ce-4609-ad17-485048a970cb/volumes"
Jul 17 10:11:25.861652 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:25.861621    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-f7gpk"]
Jul 17 10:11:26.006699 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:26.006672    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-f7gpk"
Jul 17 10:11:26.121680 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:26.121573    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b75aebb9-ae40-4709-a131-1bd2a60be619-maas-api-service-ca\") pod \"b75aebb9-ae40-4709-a131-1bd2a60be619\" (UID: \"b75aebb9-ae40-4709-a131-1bd2a60be619\") "
Jul 17 10:11:26.121945 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:26.121924    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b75aebb9-ae40-4709-a131-1bd2a60be619-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b75aebb9-ae40-4709-a131-1bd2a60be619" (UID: "b75aebb9-ae40-4709-a131-1bd2a60be619"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:11:26.222470 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:26.222434    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b75aebb9-ae40-4709-a131-1bd2a60be619-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:11:26.850423 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:26.850391    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-f7gpk"
Jul 17 10:11:26.850639 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:26.850430    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-f7gpk" event={"ID":"b75aebb9-ae40-4709-a131-1bd2a60be619","Type":"ContainerDied","Data":"aa337a0b475f200a232dca5d5346f97e0d70547b2b19d428fb04b922f29e9a60"}
Jul 17 10:11:26.850639 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:26.850504    2574 scope.go:117] "RemoveContainer" containerID="1f9139ad2944836c463cf9700a18c4853c334fdd50103556d48cb96aa35f23e9"
Jul 17 10:11:26.873413 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:26.873388    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-f7gpk"]
Jul 17 10:11:26.880066 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:26.880040    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-f7gpk"]
Jul 17 10:11:26.898254 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:26.898219    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b75aebb9-ae40-4709-a131-1bd2a60be619" path="/var/lib/kubelet/pods/b75aebb9-ae40-4709-a131-1bd2a60be619/volumes"
Jul 17 10:11:28.387852 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.387819    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-94jrv"]
Jul 17 10:11:28.388305 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.388292    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b75aebb9-ae40-4709-a131-1bd2a60be619" containerName="revoke-keys"
Jul 17 10:11:28.388305 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.388306    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b75aebb9-ae40-4709-a131-1bd2a60be619" containerName="revoke-keys"
Jul 17 10:11:28.388390 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.388319    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac90c7ba-f6ce-4609-ad17-485048a970cb" containerName="revoke-keys"
Jul 17 10:11:28.388390 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.388325    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac90c7ba-f6ce-4609-ad17-485048a970cb" containerName="revoke-keys"
Jul 17 10:11:28.388390 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.388331    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d1a1089-b418-49f5-beca-0b0f368f67ad" containerName="revoke-keys"
Jul 17 10:11:28.388390 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.388337    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d1a1089-b418-49f5-beca-0b0f368f67ad" containerName="revoke-keys"
Jul 17 10:11:28.388390 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.388358    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d1a1089-b418-49f5-beca-0b0f368f67ad" containerName="revoke-keys"
Jul 17 10:11:28.388390 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.388363    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d1a1089-b418-49f5-beca-0b0f368f67ad" containerName="revoke-keys"
Jul 17 10:11:28.388390 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.388369    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b75aebb9-ae40-4709-a131-1bd2a60be619" containerName="revoke-keys"
Jul 17 10:11:28.388390 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.388373    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b75aebb9-ae40-4709-a131-1bd2a60be619" containerName="revoke-keys"
Jul 17 10:11:28.388390 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.388379    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac90c7ba-f6ce-4609-ad17-485048a970cb" containerName="revoke-keys"
Jul 17 10:11:28.388390 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.388384    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac90c7ba-f6ce-4609-ad17-485048a970cb" containerName="revoke-keys"
Jul 17 10:11:28.388390 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.388395    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac90c7ba-f6ce-4609-ad17-485048a970cb" containerName="revoke-keys"
Jul 17 10:11:28.388706 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.388400    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac90c7ba-f6ce-4609-ad17-485048a970cb" containerName="revoke-keys"
Jul 17 10:11:28.388706 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.388460    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac90c7ba-f6ce-4609-ad17-485048a970cb" containerName="revoke-keys"
Jul 17 10:11:28.388706 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.388467    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b75aebb9-ae40-4709-a131-1bd2a60be619" containerName="revoke-keys"
Jul 17 10:11:28.388706 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.388474    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d1a1089-b418-49f5-beca-0b0f368f67ad" containerName="revoke-keys"
Jul 17 10:11:28.388706 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.388479    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac90c7ba-f6ce-4609-ad17-485048a970cb" containerName="revoke-keys"
Jul 17 10:11:28.388706 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.388486    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b75aebb9-ae40-4709-a131-1bd2a60be619" containerName="revoke-keys"
Jul 17 10:11:28.388706 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.388493    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d1a1089-b418-49f5-beca-0b0f368f67ad" containerName="revoke-keys"
Jul 17 10:11:28.388706 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.388498    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b75aebb9-ae40-4709-a131-1bd2a60be619" containerName="revoke-keys"
Jul 17 10:11:28.388706 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.388504    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d1a1089-b418-49f5-beca-0b0f368f67ad" containerName="revoke-keys"
Jul 17 10:11:28.393386 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.393367    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-94jrv"
Jul 17 10:11:28.396529 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.396431    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:11:28.399973 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.399951    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-94jrv"]
Jul 17 10:11:28.443536 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.443504    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cddbe06-8faf-4c21-b313-48c948c362e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-94jrv\" (UID: \"1cddbe06-8faf-4c21-b313-48c948c362e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-94jrv"
Jul 17 10:11:28.544976 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.544945    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cddbe06-8faf-4c21-b313-48c948c362e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-94jrv\" (UID: \"1cddbe06-8faf-4c21-b313-48c948c362e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-94jrv"
Jul 17 10:11:28.545581 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.545559    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cddbe06-8faf-4c21-b313-48c948c362e9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-94jrv\" (UID: \"1cddbe06-8faf-4c21-b313-48c948c362e9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-94jrv"
Jul 17 10:11:28.707112 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.707075    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-94jrv"
Jul 17 10:11:28.857461 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:28.857435    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-94jrv"]
Jul 17 10:11:28.859400 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:11:28.859368    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cddbe06_8faf_4c21_b313_48c948c362e9.slice/crio-0ceb7e95d3c8e49805aa889d0b9592c9c487c5bd3a48112d0557267e423392e8 WatchSource:0}: Error finding container 0ceb7e95d3c8e49805aa889d0b9592c9c487c5bd3a48112d0557267e423392e8: Status 404 returned error can't find the container with id 0ceb7e95d3c8e49805aa889d0b9592c9c487c5bd3a48112d0557267e423392e8
Jul 17 10:11:29.871411 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:29.871315    2574 generic.go:356] "Generic (PLEG): container finished" podID="1cddbe06-8faf-4c21-b313-48c948c362e9" containerID="f29e93a92a9cee4d0a608e4d80d49d10daddd2d4770696730a9116299a7856e5" exitCode=6
Jul 17 10:11:29.871765 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:29.871409    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-94jrv" event={"ID":"1cddbe06-8faf-4c21-b313-48c948c362e9","Type":"ContainerDied","Data":"f29e93a92a9cee4d0a608e4d80d49d10daddd2d4770696730a9116299a7856e5"}
Jul 17 10:11:29.871765 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:29.871471    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-94jrv" event={"ID":"1cddbe06-8faf-4c21-b313-48c948c362e9","Type":"ContainerStarted","Data":"0ceb7e95d3c8e49805aa889d0b9592c9c487c5bd3a48112d0557267e423392e8"}
Jul 17 10:11:29.871765 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:29.871582    2574 scope.go:117] "RemoveContainer" containerID="f29e93a92a9cee4d0a608e4d80d49d10daddd2d4770696730a9116299a7856e5"
Jul 17 10:11:30.879581 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:30.879545    2574 generic.go:356] "Generic (PLEG): container finished" podID="1cddbe06-8faf-4c21-b313-48c948c362e9" containerID="9c39b4f6074f1a2b3f944f6f28c75b82aa24abf0548bc731ebb29b06a0867064" exitCode=6
Jul 17 10:11:30.879993 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:30.879601    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-94jrv" event={"ID":"1cddbe06-8faf-4c21-b313-48c948c362e9","Type":"ContainerDied","Data":"9c39b4f6074f1a2b3f944f6f28c75b82aa24abf0548bc731ebb29b06a0867064"}
Jul 17 10:11:30.879993 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:30.879667    2574 scope.go:117] "RemoveContainer" containerID="f29e93a92a9cee4d0a608e4d80d49d10daddd2d4770696730a9116299a7856e5"
Jul 17 10:11:30.879993 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:30.879843    2574 scope.go:117] "RemoveContainer" containerID="9c39b4f6074f1a2b3f944f6f28c75b82aa24abf0548bc731ebb29b06a0867064"
Jul 17 10:11:30.880148 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:11:30.880116    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-94jrv_odh-ai-gateway-infra(1cddbe06-8faf-4c21-b313-48c948c362e9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-94jrv" podUID="1cddbe06-8faf-4c21-b313-48c948c362e9"
Jul 17 10:11:31.887060 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:31.887019    2574 scope.go:117] "RemoveContainer" containerID="9c39b4f6074f1a2b3f944f6f28c75b82aa24abf0548bc731ebb29b06a0867064"
Jul 17 10:11:31.887479 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:11:31.887204    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-94jrv_odh-ai-gateway-infra(1cddbe06-8faf-4c21-b313-48c948c362e9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-94jrv" podUID="1cddbe06-8faf-4c21-b313-48c948c362e9"
Jul 17 10:11:43.138065 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:43.137948    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dh564_84db06ab-87e2-43b3-82d0-13f06ba86d06/ovn-acl-logging/0.log"
Jul 17 10:11:43.144304 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:43.144287    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dh564_84db06ab-87e2-43b3-82d0-13f06ba86d06/ovn-acl-logging/0.log"
Jul 17 10:11:43.892320 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:43.892216    2574 scope.go:117] "RemoveContainer" containerID="9c39b4f6074f1a2b3f944f6f28c75b82aa24abf0548bc731ebb29b06a0867064"
Jul 17 10:11:44.958341 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:44.958309    2574 generic.go:356] "Generic (PLEG): container finished" podID="1cddbe06-8faf-4c21-b313-48c948c362e9" containerID="1b9771eade63c742db654bb8ea4008123e51cccae10d36a7967e61c7efed8abe" exitCode=6
Jul 17 10:11:44.958750 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:44.958390    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-94jrv" event={"ID":"1cddbe06-8faf-4c21-b313-48c948c362e9","Type":"ContainerDied","Data":"1b9771eade63c742db654bb8ea4008123e51cccae10d36a7967e61c7efed8abe"}
Jul 17 10:11:44.958750 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:44.958452    2574 scope.go:117] "RemoveContainer" containerID="9c39b4f6074f1a2b3f944f6f28c75b82aa24abf0548bc731ebb29b06a0867064"
Jul 17 10:11:44.958750 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:44.958691    2574 scope.go:117] "RemoveContainer" containerID="1b9771eade63c742db654bb8ea4008123e51cccae10d36a7967e61c7efed8abe"
Jul 17 10:11:44.958946 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:11:44.958929    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-94jrv_odh-ai-gateway-infra(1cddbe06-8faf-4c21-b313-48c948c362e9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-94jrv" podUID="1cddbe06-8faf-4c21-b313-48c948c362e9"
Jul 17 10:11:45.986229 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:45.986197    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-94jrv"]
Jul 17 10:11:46.132276 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:46.132231    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-94jrv"
Jul 17 10:11:46.223302 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:46.223268    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cddbe06-8faf-4c21-b313-48c948c362e9-maas-api-service-ca\") pod \"1cddbe06-8faf-4c21-b313-48c948c362e9\" (UID: \"1cddbe06-8faf-4c21-b313-48c948c362e9\") "
Jul 17 10:11:46.223663 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:46.223638    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cddbe06-8faf-4c21-b313-48c948c362e9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1cddbe06-8faf-4c21-b313-48c948c362e9" (UID: "1cddbe06-8faf-4c21-b313-48c948c362e9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:11:46.324165 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:46.324087    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cddbe06-8faf-4c21-b313-48c948c362e9-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:11:46.979725 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:46.979694    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-94jrv"
Jul 17 10:11:46.979725 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:46.979704    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-94jrv" event={"ID":"1cddbe06-8faf-4c21-b313-48c948c362e9","Type":"ContainerDied","Data":"0ceb7e95d3c8e49805aa889d0b9592c9c487c5bd3a48112d0557267e423392e8"}
Jul 17 10:11:46.979971 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:46.979769    2574 scope.go:117] "RemoveContainer" containerID="1b9771eade63c742db654bb8ea4008123e51cccae10d36a7967e61c7efed8abe"
Jul 17 10:11:46.999906 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:46.999873    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-94jrv"]
Jul 17 10:11:47.009736 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:47.009708    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-94jrv"]
Jul 17 10:11:48.897503 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:48.897469    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cddbe06-8faf-4c21-b313-48c948c362e9" path="/var/lib/kubelet/pods/1cddbe06-8faf-4c21-b313-48c948c362e9/volumes"
Jul 17 10:11:55.681467 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:55.681436    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-z5ksr"]
Jul 17 10:11:55.681896 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:55.681883    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d1a1089-b418-49f5-beca-0b0f368f67ad" containerName="revoke-keys"
Jul 17 10:11:55.681943 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:55.681897    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d1a1089-b418-49f5-beca-0b0f368f67ad" containerName="revoke-keys"
Jul 17 10:11:55.681943 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:55.681907    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cddbe06-8faf-4c21-b313-48c948c362e9" containerName="revoke-keys"
Jul 17 10:11:55.681943 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:55.681913    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cddbe06-8faf-4c21-b313-48c948c362e9" containerName="revoke-keys"
Jul 17 10:11:55.681943 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:55.681919    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cddbe06-8faf-4c21-b313-48c948c362e9" containerName="revoke-keys"
Jul 17 10:11:55.681943 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:55.681925    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cddbe06-8faf-4c21-b313-48c948c362e9" containerName="revoke-keys"
Jul 17 10:11:55.681943 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:55.681937    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b75aebb9-ae40-4709-a131-1bd2a60be619" containerName="revoke-keys"
Jul 17 10:11:55.681943 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:55.681942    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b75aebb9-ae40-4709-a131-1bd2a60be619" containerName="revoke-keys"
Jul 17 10:11:55.682158 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:55.681949    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cddbe06-8faf-4c21-b313-48c948c362e9" containerName="revoke-keys"
Jul 17 10:11:55.682158 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:55.681954    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cddbe06-8faf-4c21-b313-48c948c362e9" containerName="revoke-keys"
Jul 17 10:11:55.682158 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:55.682015    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cddbe06-8faf-4c21-b313-48c948c362e9" containerName="revoke-keys"
Jul 17 10:11:55.682158 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:55.682025    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac90c7ba-f6ce-4609-ad17-485048a970cb" containerName="revoke-keys"
Jul 17 10:11:55.682158 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:55.682034    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cddbe06-8faf-4c21-b313-48c948c362e9" containerName="revoke-keys"
Jul 17 10:11:55.685251 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:55.685225    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-z5ksr"
Jul 17 10:11:55.688363 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:55.688338    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:11:55.691347 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:55.691324    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-z5ksr"]
Jul 17 10:11:55.709845 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:55.709814    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/508a18e6-750e-4346-a9d3-39e96ae42ffc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-z5ksr\" (UID: \"508a18e6-750e-4346-a9d3-39e96ae42ffc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-z5ksr"
Jul 17 10:11:55.810807 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:55.810762    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/508a18e6-750e-4346-a9d3-39e96ae42ffc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-z5ksr\" (UID: \"508a18e6-750e-4346-a9d3-39e96ae42ffc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-z5ksr"
Jul 17 10:11:55.811433 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:55.811413    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/508a18e6-750e-4346-a9d3-39e96ae42ffc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-z5ksr\" (UID: \"508a18e6-750e-4346-a9d3-39e96ae42ffc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-z5ksr"
Jul 17 10:11:55.998520 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:55.998436    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-z5ksr"
Jul 17 10:11:56.148170 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:56.148145    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-z5ksr"]
Jul 17 10:11:56.153546 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:11:56.153514    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod508a18e6_750e_4346_a9d3_39e96ae42ffc.slice/crio-d7b774599461095dde5b81d7321f78f93c0d5d412f28d7e02712c1d1186aaf06 WatchSource:0}: Error finding container d7b774599461095dde5b81d7321f78f93c0d5d412f28d7e02712c1d1186aaf06: Status 404 returned error can't find the container with id d7b774599461095dde5b81d7321f78f93c0d5d412f28d7e02712c1d1186aaf06
Jul 17 10:11:56.277912 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:56.277815    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hb6vt"]
Jul 17 10:11:56.278454 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:56.278433    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cddbe06-8faf-4c21-b313-48c948c362e9" containerName="revoke-keys"
Jul 17 10:11:56.282640 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:56.282620    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hb6vt"
Jul 17 10:11:56.288987 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:56.288962    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hb6vt"]
Jul 17 10:11:56.316546 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:56.316514    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a39b303-3b88-4e50-99d6-0282dcc32c6a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-hb6vt\" (UID: \"0a39b303-3b88-4e50-99d6-0282dcc32c6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hb6vt"
Jul 17 10:11:56.417627 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:56.417602    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a39b303-3b88-4e50-99d6-0282dcc32c6a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-hb6vt\" (UID: \"0a39b303-3b88-4e50-99d6-0282dcc32c6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hb6vt"
Jul 17 10:11:56.418175 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:56.418157    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a39b303-3b88-4e50-99d6-0282dcc32c6a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-hb6vt\" (UID: \"0a39b303-3b88-4e50-99d6-0282dcc32c6a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hb6vt"
Jul 17 10:11:56.595419 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:56.595322    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hb6vt"
Jul 17 10:11:56.962888 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:56.962854    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hb6vt"]
Jul 17 10:11:56.964721 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:11:56.964686    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a39b303_3b88_4e50_99d6_0282dcc32c6a.slice/crio-e9a6659aed9a2035eefb9c36a88bbcaf34fe3a52decc5e5323c90835c051095b WatchSource:0}: Error finding container e9a6659aed9a2035eefb9c36a88bbcaf34fe3a52decc5e5323c90835c051095b: Status 404 returned error can't find the container with id e9a6659aed9a2035eefb9c36a88bbcaf34fe3a52decc5e5323c90835c051095b
Jul 17 10:11:57.037688 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:57.037654    2574 generic.go:356] "Generic (PLEG): container finished" podID="508a18e6-750e-4346-a9d3-39e96ae42ffc" containerID="f3f8efca8cf37afd27812a0868c49eeef1708f40418d543eb2e195aa52df44fe" exitCode=6
Jul 17 10:11:57.037874 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:57.037739    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-z5ksr" event={"ID":"508a18e6-750e-4346-a9d3-39e96ae42ffc","Type":"ContainerDied","Data":"f3f8efca8cf37afd27812a0868c49eeef1708f40418d543eb2e195aa52df44fe"}
Jul 17 10:11:57.037874 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:57.037801    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-z5ksr" event={"ID":"508a18e6-750e-4346-a9d3-39e96ae42ffc","Type":"ContainerStarted","Data":"d7b774599461095dde5b81d7321f78f93c0d5d412f28d7e02712c1d1186aaf06"}
Jul 17 10:11:57.038007 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:57.037967    2574 scope.go:117] "RemoveContainer" containerID="f3f8efca8cf37afd27812a0868c49eeef1708f40418d543eb2e195aa52df44fe"
Jul 17 10:11:57.039286 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:57.039259    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hb6vt" event={"ID":"0a39b303-3b88-4e50-99d6-0282dcc32c6a","Type":"ContainerStarted","Data":"e9a6659aed9a2035eefb9c36a88bbcaf34fe3a52decc5e5323c90835c051095b"}
Jul 17 10:11:57.306781 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:57.306745    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-r9xtt"]
Jul 17 10:11:57.310468 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:57.310446    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-r9xtt"
Jul 17 10:11:57.317882 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:57.317795    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-r9xtt"]
Jul 17 10:11:57.326155 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:57.326118    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-r9xtt\" (UID: \"b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-r9xtt"
Jul 17 10:11:57.426930 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:57.426878    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-r9xtt\" (UID: \"b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-r9xtt"
Jul 17 10:11:57.427570 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:57.427550    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-r9xtt\" (UID: \"b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-r9xtt"
Jul 17 10:11:57.624547 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:57.624524    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-r9xtt"
Jul 17 10:11:57.765811 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:57.765784    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-r9xtt"]
Jul 17 10:11:57.767985 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:11:57.767959    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb518de5f_3fd4_45a0_8a64_b7ee4bc1c11e.slice/crio-8e6868892fd2a2b56b2dd6ca75242b7f58cc3e0fda59b1e1db295446a4af0684 WatchSource:0}: Error finding container 8e6868892fd2a2b56b2dd6ca75242b7f58cc3e0fda59b1e1db295446a4af0684: Status 404 returned error can't find the container with id 8e6868892fd2a2b56b2dd6ca75242b7f58cc3e0fda59b1e1db295446a4af0684
Jul 17 10:11:58.046630 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:58.046590    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-r9xtt" event={"ID":"b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e","Type":"ContainerStarted","Data":"8e6868892fd2a2b56b2dd6ca75242b7f58cc3e0fda59b1e1db295446a4af0684"}
Jul 17 10:11:58.048549 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:58.048525    2574 generic.go:356] "Generic (PLEG): container finished" podID="508a18e6-750e-4346-a9d3-39e96ae42ffc" containerID="2eed085bda669e7e264fefa15871a121ac2b8fb561d30f7d21ac275cea52794a" exitCode=6
Jul 17 10:11:58.048682 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:58.048609    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-z5ksr" event={"ID":"508a18e6-750e-4346-a9d3-39e96ae42ffc","Type":"ContainerDied","Data":"2eed085bda669e7e264fefa15871a121ac2b8fb561d30f7d21ac275cea52794a"}
Jul 17 10:11:58.048682 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:58.048667    2574 scope.go:117] "RemoveContainer" containerID="f3f8efca8cf37afd27812a0868c49eeef1708f40418d543eb2e195aa52df44fe"
Jul 17 10:11:58.048924 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:58.048895    2574 scope.go:117] "RemoveContainer" containerID="2eed085bda669e7e264fefa15871a121ac2b8fb561d30f7d21ac275cea52794a"
Jul 17 10:11:58.049205 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:11:58.049180    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-z5ksr_odh-ai-gateway-infra(508a18e6-750e-4346-a9d3-39e96ae42ffc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-z5ksr" podUID="508a18e6-750e-4346-a9d3-39e96ae42ffc"
Jul 17 10:11:58.050353 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:58.050332    2574 generic.go:356] "Generic (PLEG): container finished" podID="0a39b303-3b88-4e50-99d6-0282dcc32c6a" containerID="c58185c73ea264ab5d8d9da80b5422f2cac5f7c510248568f83e7e6079c51326" exitCode=6
Jul 17 10:11:58.050444 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:58.050384    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hb6vt" event={"ID":"0a39b303-3b88-4e50-99d6-0282dcc32c6a","Type":"ContainerDied","Data":"c58185c73ea264ab5d8d9da80b5422f2cac5f7c510248568f83e7e6079c51326"}
Jul 17 10:11:58.050555 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:58.050543    2574 scope.go:117] "RemoveContainer" containerID="c58185c73ea264ab5d8d9da80b5422f2cac5f7c510248568f83e7e6079c51326"
Jul 17 10:11:59.057569 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:59.057481    2574 generic.go:356] "Generic (PLEG): container finished" podID="b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e" containerID="f005eef393185a9dfc1f852d1712a795aff5a04aedd8334e0ee358151f3cb0e1" exitCode=6
Jul 17 10:11:59.057996 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:59.057562    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-r9xtt" event={"ID":"b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e","Type":"ContainerDied","Data":"f005eef393185a9dfc1f852d1712a795aff5a04aedd8334e0ee358151f3cb0e1"}
Jul 17 10:11:59.057996 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:59.057752    2574 scope.go:117] "RemoveContainer" containerID="f005eef393185a9dfc1f852d1712a795aff5a04aedd8334e0ee358151f3cb0e1"
Jul 17 10:11:59.059574 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:59.059558    2574 scope.go:117] "RemoveContainer" containerID="2eed085bda669e7e264fefa15871a121ac2b8fb561d30f7d21ac275cea52794a"
Jul 17 10:11:59.059773 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:11:59.059749    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-z5ksr_odh-ai-gateway-infra(508a18e6-750e-4346-a9d3-39e96ae42ffc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-z5ksr" podUID="508a18e6-750e-4346-a9d3-39e96ae42ffc"
Jul 17 10:11:59.061275 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:59.061233    2574 generic.go:356] "Generic (PLEG): container finished" podID="0a39b303-3b88-4e50-99d6-0282dcc32c6a" containerID="ea3a71de60aa771cf237d5d879359175039ea1a2c174018728eb817ea2ea9471" exitCode=6
Jul 17 10:11:59.061396 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:59.061280    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hb6vt" event={"ID":"0a39b303-3b88-4e50-99d6-0282dcc32c6a","Type":"ContainerDied","Data":"ea3a71de60aa771cf237d5d879359175039ea1a2c174018728eb817ea2ea9471"}
Jul 17 10:11:59.061396 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:59.061340    2574 scope.go:117] "RemoveContainer" containerID="c58185c73ea264ab5d8d9da80b5422f2cac5f7c510248568f83e7e6079c51326"
Jul 17 10:11:59.061562 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:11:59.061531    2574 scope.go:117] "RemoveContainer" containerID="ea3a71de60aa771cf237d5d879359175039ea1a2c174018728eb817ea2ea9471"
Jul 17 10:11:59.061727 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:11:59.061709    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-hb6vt_odh-ai-gateway-infra(0a39b303-3b88-4e50-99d6-0282dcc32c6a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hb6vt" podUID="0a39b303-3b88-4e50-99d6-0282dcc32c6a"
Jul 17 10:12:00.069677 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:00.069641    2574 scope.go:117] "RemoveContainer" containerID="ea3a71de60aa771cf237d5d879359175039ea1a2c174018728eb817ea2ea9471"
Jul 17 10:12:00.070145 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:12:00.069936    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-hb6vt_odh-ai-gateway-infra(0a39b303-3b88-4e50-99d6-0282dcc32c6a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hb6vt" podUID="0a39b303-3b88-4e50-99d6-0282dcc32c6a"
Jul 17 10:12:00.071183 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:00.071152    2574 generic.go:356] "Generic (PLEG): container finished" podID="b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e" containerID="242fad52f6a83c8e8e3c431e4f5909032ed79c6047a9a24cead153dc6f58edb3" exitCode=6
Jul 17 10:12:00.071316 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:00.071234    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-r9xtt" event={"ID":"b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e","Type":"ContainerDied","Data":"242fad52f6a83c8e8e3c431e4f5909032ed79c6047a9a24cead153dc6f58edb3"}
Jul 17 10:12:00.071371 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:00.071327    2574 scope.go:117] "RemoveContainer" containerID="f005eef393185a9dfc1f852d1712a795aff5a04aedd8334e0ee358151f3cb0e1"
Jul 17 10:12:00.071425 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:00.071409    2574 scope.go:117] "RemoveContainer" containerID="242fad52f6a83c8e8e3c431e4f5909032ed79c6047a9a24cead153dc6f58edb3"
Jul 17 10:12:00.071614 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:12:00.071595    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-r9xtt_odh-ai-gateway-infra(b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-r9xtt" podUID="b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e"
Jul 17 10:12:01.078673 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:01.078640    2574 scope.go:117] "RemoveContainer" containerID="242fad52f6a83c8e8e3c431e4f5909032ed79c6047a9a24cead153dc6f58edb3"
Jul 17 10:12:01.079153 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:12:01.078914    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-r9xtt_odh-ai-gateway-infra(b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-r9xtt" podUID="b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e"
Jul 17 10:12:09.891953 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:09.891913    2574 scope.go:117] "RemoveContainer" containerID="2eed085bda669e7e264fefa15871a121ac2b8fb561d30f7d21ac275cea52794a"
Jul 17 10:12:10.891749 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:10.891666    2574 scope.go:117] "RemoveContainer" containerID="ea3a71de60aa771cf237d5d879359175039ea1a2c174018728eb817ea2ea9471"
Jul 17 10:12:11.133661 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:11.133627    2574 generic.go:356] "Generic (PLEG): container finished" podID="508a18e6-750e-4346-a9d3-39e96ae42ffc" containerID="5ba7ca3ee9ee6b65c60601fedafc6639bceb62eb785350326f0f7d0f0e762421" exitCode=6
Jul 17 10:12:11.134070 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:11.133708    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-z5ksr" event={"ID":"508a18e6-750e-4346-a9d3-39e96ae42ffc","Type":"ContainerDied","Data":"5ba7ca3ee9ee6b65c60601fedafc6639bceb62eb785350326f0f7d0f0e762421"}
Jul 17 10:12:11.134070 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:11.133773    2574 scope.go:117] "RemoveContainer" containerID="2eed085bda669e7e264fefa15871a121ac2b8fb561d30f7d21ac275cea52794a"
Jul 17 10:12:11.134208 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:11.134121    2574 scope.go:117] "RemoveContainer" containerID="5ba7ca3ee9ee6b65c60601fedafc6639bceb62eb785350326f0f7d0f0e762421"
Jul 17 10:12:11.134411 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:12:11.134373    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-z5ksr_odh-ai-gateway-infra(508a18e6-750e-4346-a9d3-39e96ae42ffc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-z5ksr" podUID="508a18e6-750e-4346-a9d3-39e96ae42ffc"
Jul 17 10:12:12.142686 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:12.142655    2574 generic.go:356] "Generic (PLEG): container finished" podID="0a39b303-3b88-4e50-99d6-0282dcc32c6a" containerID="f2ada3021630963660c66087e10144900d52183323d5ed4f37ea1b6878f334fc" exitCode=6
Jul 17 10:12:12.143132 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:12.142730    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hb6vt" event={"ID":"0a39b303-3b88-4e50-99d6-0282dcc32c6a","Type":"ContainerDied","Data":"f2ada3021630963660c66087e10144900d52183323d5ed4f37ea1b6878f334fc"}
Jul 17 10:12:12.143132 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:12.142793    2574 scope.go:117] "RemoveContainer" containerID="ea3a71de60aa771cf237d5d879359175039ea1a2c174018728eb817ea2ea9471"
Jul 17 10:12:12.143249 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:12.143156    2574 scope.go:117] "RemoveContainer" containerID="f2ada3021630963660c66087e10144900d52183323d5ed4f37ea1b6878f334fc"
Jul 17 10:12:12.143368 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:12:12.143350    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-hb6vt_odh-ai-gateway-infra(0a39b303-3b88-4e50-99d6-0282dcc32c6a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hb6vt" podUID="0a39b303-3b88-4e50-99d6-0282dcc32c6a"
Jul 17 10:12:12.167640 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:12.167616    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-z5ksr"]
Jul 17 10:12:12.316674 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:12.316651    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-z5ksr"
Jul 17 10:12:12.375301 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:12.375271    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/508a18e6-750e-4346-a9d3-39e96ae42ffc-maas-api-service-ca\") pod \"508a18e6-750e-4346-a9d3-39e96ae42ffc\" (UID: \"508a18e6-750e-4346-a9d3-39e96ae42ffc\") "
Jul 17 10:12:12.375589 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:12.375569    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/508a18e6-750e-4346-a9d3-39e96ae42ffc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "508a18e6-750e-4346-a9d3-39e96ae42ffc" (UID: "508a18e6-750e-4346-a9d3-39e96ae42ffc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:12:12.476634 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:12.476595    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/508a18e6-750e-4346-a9d3-39e96ae42ffc-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:12:13.149470 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:13.149441    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-z5ksr"
Jul 17 10:12:13.149969 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:13.149472    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-z5ksr" event={"ID":"508a18e6-750e-4346-a9d3-39e96ae42ffc","Type":"ContainerDied","Data":"d7b774599461095dde5b81d7321f78f93c0d5d412f28d7e02712c1d1186aaf06"}
Jul 17 10:12:13.149969 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:13.149542    2574 scope.go:117] "RemoveContainer" containerID="5ba7ca3ee9ee6b65c60601fedafc6639bceb62eb785350326f0f7d0f0e762421"
Jul 17 10:12:13.168933 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:13.168856    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-z5ksr"]
Jul 17 10:12:13.172228 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:13.172205    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-z5ksr"]
Jul 17 10:12:13.174076 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:13.174054    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hb6vt"]
Jul 17 10:12:13.316690 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:13.316664    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hb6vt"
Jul 17 10:12:13.385143 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:13.385109    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a39b303-3b88-4e50-99d6-0282dcc32c6a-maas-api-service-ca\") pod \"0a39b303-3b88-4e50-99d6-0282dcc32c6a\" (UID: \"0a39b303-3b88-4e50-99d6-0282dcc32c6a\") "
Jul 17 10:12:13.385465 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:13.385443    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a39b303-3b88-4e50-99d6-0282dcc32c6a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0a39b303-3b88-4e50-99d6-0282dcc32c6a" (UID: "0a39b303-3b88-4e50-99d6-0282dcc32c6a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:12:13.486285 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:13.486177    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a39b303-3b88-4e50-99d6-0282dcc32c6a-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:12:14.158713 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:14.158687    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hb6vt"
Jul 17 10:12:14.159180 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:14.158682    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hb6vt" event={"ID":"0a39b303-3b88-4e50-99d6-0282dcc32c6a","Type":"ContainerDied","Data":"e9a6659aed9a2035eefb9c36a88bbcaf34fe3a52decc5e5323c90835c051095b"}
Jul 17 10:12:14.159180 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:14.158829    2574 scope.go:117] "RemoveContainer" containerID="f2ada3021630963660c66087e10144900d52183323d5ed4f37ea1b6878f334fc"
Jul 17 10:12:14.183991 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:14.183958    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hb6vt"]
Jul 17 10:12:14.190610 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:14.190586    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hb6vt"]
Jul 17 10:12:14.898422 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:14.898387    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a39b303-3b88-4e50-99d6-0282dcc32c6a" path="/var/lib/kubelet/pods/0a39b303-3b88-4e50-99d6-0282dcc32c6a/volumes"
Jul 17 10:12:14.898792 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:14.898778    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="508a18e6-750e-4346-a9d3-39e96ae42ffc" path="/var/lib/kubelet/pods/508a18e6-750e-4346-a9d3-39e96ae42ffc/volumes"
Jul 17 10:12:15.188502 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.188463    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"]
Jul 17 10:12:15.189174 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.189157    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a39b303-3b88-4e50-99d6-0282dcc32c6a" containerName="revoke-keys"
Jul 17 10:12:15.189218 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.189177    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a39b303-3b88-4e50-99d6-0282dcc32c6a" containerName="revoke-keys"
Jul 17 10:12:15.189218 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.189190    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="508a18e6-750e-4346-a9d3-39e96ae42ffc" containerName="revoke-keys"
Jul 17 10:12:15.189218 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.189199    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="508a18e6-750e-4346-a9d3-39e96ae42ffc" containerName="revoke-keys"
Jul 17 10:12:15.189218 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.189211    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="508a18e6-750e-4346-a9d3-39e96ae42ffc" containerName="revoke-keys"
Jul 17 10:12:15.189374 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.189220    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="508a18e6-750e-4346-a9d3-39e96ae42ffc" containerName="revoke-keys"
Jul 17 10:12:15.189374 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.189269    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a39b303-3b88-4e50-99d6-0282dcc32c6a" containerName="revoke-keys"
Jul 17 10:12:15.189374 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.189280    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a39b303-3b88-4e50-99d6-0282dcc32c6a" containerName="revoke-keys"
Jul 17 10:12:15.189473 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.189384    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a39b303-3b88-4e50-99d6-0282dcc32c6a" containerName="revoke-keys"
Jul 17 10:12:15.189473 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.189394    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="508a18e6-750e-4346-a9d3-39e96ae42ffc" containerName="revoke-keys"
Jul 17 10:12:15.189473 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.189405    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="508a18e6-750e-4346-a9d3-39e96ae42ffc" containerName="revoke-keys"
Jul 17 10:12:15.189473 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.189416    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a39b303-3b88-4e50-99d6-0282dcc32c6a" containerName="revoke-keys"
Jul 17 10:12:15.194705 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.194681    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.197599 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.197576    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-1c996b-openshift-default-dockercfg-hh2rg\""
Jul 17 10:12:15.206228 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.206206    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"]
Jul 17 10:12:15.302682 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.302642    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfpq8\" (UniqueName: \"kubernetes.io/projected/8b0de510-d71b-4b55-8217-3c9b8428668b-kube-api-access-bfpq8\") pod \"e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") " pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.302682 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.302697    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-istio-data\") pod \"e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") " pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.302962 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.302730    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-credential-socket\") pod \"e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") " pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.302962 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.302796    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b0de510-d71b-4b55-8217-3c9b8428668b-istiod-ca-cert\") pod \"e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") " pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.302962 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.302881    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b0de510-d71b-4b55-8217-3c9b8428668b-istio-token\") pod \"e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") " pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.303103 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.302972    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b0de510-d71b-4b55-8217-3c9b8428668b-istio-podinfo\") pod \"e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") " pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.303103 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.303078    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-istio-envoy\") pod \"e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") " pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.303192 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.303163    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-workload-certs\") pod \"e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") " pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.303277 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.303217    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-workload-socket\") pod \"e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") " pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.403930 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.403892    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-istio-envoy\") pod \"e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") " pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.404125 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.403953    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-workload-certs\") pod \"e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") " pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.404125 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.403986    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-workload-socket\") pod \"e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") " pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.404125 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.404025    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfpq8\" (UniqueName: \"kubernetes.io/projected/8b0de510-d71b-4b55-8217-3c9b8428668b-kube-api-access-bfpq8\") pod \"e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") " pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.404125 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.404051    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-istio-data\") pod \"e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") " pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.404367 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.404198    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-credential-socket\") pod \"e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") " pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.404367 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.404275    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b0de510-d71b-4b55-8217-3c9b8428668b-istiod-ca-cert\") pod \"e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") " pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.404367 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.404347    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b0de510-d71b-4b55-8217-3c9b8428668b-istio-token\") pod \"e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") " pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.404525 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.404380    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b0de510-d71b-4b55-8217-3c9b8428668b-istio-podinfo\") pod \"e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") " pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.404525 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.404460    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-workload-certs\") pod \"e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") " pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.404755 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.404732    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-credential-socket\") pod \"e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") " pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.404755 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.404745    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-istio-data\") pod \"e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") " pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.404893 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.404827    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-workload-socket\") pod \"e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") " pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.405354 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.405322    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b0de510-d71b-4b55-8217-3c9b8428668b-istiod-ca-cert\") pod \"e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") " pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.406601 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.406576    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-istio-envoy\") pod \"e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") " pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.407115 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.407090    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b0de510-d71b-4b55-8217-3c9b8428668b-istio-podinfo\") pod \"e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") " pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.413056 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.413031    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b0de510-d71b-4b55-8217-3c9b8428668b-istio-token\") pod \"e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") " pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.413270 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.413225    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfpq8\" (UniqueName: \"kubernetes.io/projected/8b0de510-d71b-4b55-8217-3c9b8428668b-kube-api-access-bfpq8\") pod \"e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") " pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.509674 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.509590    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:15.662326 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.662299    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"]
Jul 17 10:12:15.663496 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:12:15.663457    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b0de510_d71b_4b55_8217_3c9b8428668b.slice/crio-eff4bfda427667784ad30b70b39da3e9d6186407e39f339bab00a01a0020db6d WatchSource:0}: Error finding container eff4bfda427667784ad30b70b39da3e9d6186407e39f339bab00a01a0020db6d: Status 404 returned error can't find the container with id eff4bfda427667784ad30b70b39da3e9d6186407e39f339bab00a01a0020db6d
Jul 17 10:12:15.665988 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.665955    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 10:12:15.666054 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.666028    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 10:12:15.666091 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:15.666056    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 10:12:16.172703 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:16.172664    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl" event={"ID":"8b0de510-d71b-4b55-8217-3c9b8428668b","Type":"ContainerStarted","Data":"aaa0321094ca1b90a363d9e2f6a4266e151d629cf411dec23c9df9b95bb8f7e0"}
Jul 17 10:12:16.172908 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:16.172721    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl" event={"ID":"8b0de510-d71b-4b55-8217-3c9b8428668b","Type":"ContainerStarted","Data":"eff4bfda427667784ad30b70b39da3e9d6186407e39f339bab00a01a0020db6d"}
Jul 17 10:12:16.194060 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:16.194012    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl" podStartSLOduration=1.193996596 podStartE2EDuration="1.193996596s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:12:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:12:16.19247682 +0000 UTC m=+2733.775115197" watchObservedRunningTime="2026-07-17 10:12:16.193996596 +0000 UTC m=+2733.776634972"
Jul 17 10:12:16.510697 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:16.510609    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:16.516018 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:16.515986    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:16.892142 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:16.892042    2574 scope.go:117] "RemoveContainer" containerID="242fad52f6a83c8e8e3c431e4f5909032ed79c6047a9a24cead153dc6f58edb3"
Jul 17 10:12:17.178669 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:17.178625    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:17.179630 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:17.179610    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:18.187380 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:18.187350    2574 generic.go:356] "Generic (PLEG): container finished" podID="b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e" containerID="e6b7771571f91aebcf9dc42b8f5eafa9bf03677f9d2c8adfdb7028d46bd0aa95" exitCode=6
Jul 17 10:12:18.187794 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:18.187417    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-r9xtt" event={"ID":"b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e","Type":"ContainerDied","Data":"e6b7771571f91aebcf9dc42b8f5eafa9bf03677f9d2c8adfdb7028d46bd0aa95"}
Jul 17 10:12:18.187794 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:18.187476    2574 scope.go:117] "RemoveContainer" containerID="242fad52f6a83c8e8e3c431e4f5909032ed79c6047a9a24cead153dc6f58edb3"
Jul 17 10:12:18.187937 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:18.187864    2574 scope.go:117] "RemoveContainer" containerID="e6b7771571f91aebcf9dc42b8f5eafa9bf03677f9d2c8adfdb7028d46bd0aa95"
Jul 17 10:12:18.188122 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:12:18.188097    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-r9xtt_odh-ai-gateway-infra(b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-r9xtt" podUID="b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e"
Jul 17 10:12:18.462619 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:18.462520    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7"]
Jul 17 10:12:18.463117 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:18.463097    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a39b303-3b88-4e50-99d6-0282dcc32c6a" containerName="revoke-keys"
Jul 17 10:12:18.463230 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:18.463120    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a39b303-3b88-4e50-99d6-0282dcc32c6a" containerName="revoke-keys"
Jul 17 10:12:18.463230 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:18.463132    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="508a18e6-750e-4346-a9d3-39e96ae42ffc" containerName="revoke-keys"
Jul 17 10:12:18.463230 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:18.463140    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="508a18e6-750e-4346-a9d3-39e96ae42ffc" containerName="revoke-keys"
Jul 17 10:12:18.463420 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:18.463263    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a39b303-3b88-4e50-99d6-0282dcc32c6a" containerName="revoke-keys"
Jul 17 10:12:18.463420 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:18.463283    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="508a18e6-750e-4346-a9d3-39e96ae42ffc" containerName="revoke-keys"
Jul 17 10:12:18.467994 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:18.467969    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7"
Jul 17 10:12:18.478487 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:18.478458    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7"]
Jul 17 10:12:18.537718 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:18.537678    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a858656-ad84-4466-88ad-1da91ab57178-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7\" (UID: \"0a858656-ad84-4466-88ad-1da91ab57178\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7"
Jul 17 10:12:18.639193 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:18.639162    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a858656-ad84-4466-88ad-1da91ab57178-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7\" (UID: \"0a858656-ad84-4466-88ad-1da91ab57178\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7"
Jul 17 10:12:18.639870 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:18.639842    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a858656-ad84-4466-88ad-1da91ab57178-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7\" (UID: \"0a858656-ad84-4466-88ad-1da91ab57178\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7"
Jul 17 10:12:18.780058 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:18.779980    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7"
Jul 17 10:12:18.965625 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:18.965597    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7"]
Jul 17 10:12:18.966420 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:12:18.966383    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a858656_ad84_4466_88ad_1da91ab57178.slice/crio-9cfb35f451608272f7afbcaf5cd5967d6baaedc4f25886645cfef788eb9ae9e2 WatchSource:0}: Error finding container 9cfb35f451608272f7afbcaf5cd5967d6baaedc4f25886645cfef788eb9ae9e2: Status 404 returned error can't find the container with id 9cfb35f451608272f7afbcaf5cd5967d6baaedc4f25886645cfef788eb9ae9e2
Jul 17 10:12:19.195394 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:19.195358    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7" event={"ID":"0a858656-ad84-4466-88ad-1da91ab57178","Type":"ContainerStarted","Data":"9cfb35f451608272f7afbcaf5cd5967d6baaedc4f25886645cfef788eb9ae9e2"}
Jul 17 10:12:19.234910 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:19.234875    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-r9xtt"]
Jul 17 10:12:19.371415 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:19.371392    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-r9xtt"
Jul 17 10:12:19.448617 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:19.448526    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e-maas-api-service-ca\") pod \"b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e\" (UID: \"b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e\") "
Jul 17 10:12:19.448931 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:19.448905    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e" (UID: "b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:12:19.549510 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:19.549471    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:12:20.206876 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:20.206832    2574 generic.go:356] "Generic (PLEG): container finished" podID="0a858656-ad84-4466-88ad-1da91ab57178" containerID="f281cb1b715a331f578e7486656f3e4efae3f005ebb49297dfba5e53b85b67c5" exitCode=6
Jul 17 10:12:20.207396 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:20.206930    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7" event={"ID":"0a858656-ad84-4466-88ad-1da91ab57178","Type":"ContainerDied","Data":"f281cb1b715a331f578e7486656f3e4efae3f005ebb49297dfba5e53b85b67c5"}
Jul 17 10:12:20.207396 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:20.207207    2574 scope.go:117] "RemoveContainer" containerID="f281cb1b715a331f578e7486656f3e4efae3f005ebb49297dfba5e53b85b67c5"
Jul 17 10:12:20.209333 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:20.209303    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-r9xtt"
Jul 17 10:12:20.209473 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:20.209345    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-r9xtt" event={"ID":"b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e","Type":"ContainerDied","Data":"8e6868892fd2a2b56b2dd6ca75242b7f58cc3e0fda59b1e1db295446a4af0684"}
Jul 17 10:12:20.209473 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:20.209399    2574 scope.go:117] "RemoveContainer" containerID="e6b7771571f91aebcf9dc42b8f5eafa9bf03677f9d2c8adfdb7028d46bd0aa95"
Jul 17 10:12:20.246690 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:20.246667    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-r9xtt"]
Jul 17 10:12:20.251735 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:20.251713    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-r9xtt"]
Jul 17 10:12:20.897800 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:20.897722    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e" path="/var/lib/kubelet/pods/b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e/volumes"
Jul 17 10:12:21.217567 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:21.217534    2574 generic.go:356] "Generic (PLEG): container finished" podID="0a858656-ad84-4466-88ad-1da91ab57178" containerID="68ecb6a1a7856d0bff743fc2ff21d9624ad99203e99774ccb733e9ae3d1ac97b" exitCode=6
Jul 17 10:12:21.218048 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:21.217617    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7" event={"ID":"0a858656-ad84-4466-88ad-1da91ab57178","Type":"ContainerDied","Data":"68ecb6a1a7856d0bff743fc2ff21d9624ad99203e99774ccb733e9ae3d1ac97b"}
Jul 17 10:12:21.218048 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:21.217672    2574 scope.go:117] "RemoveContainer" containerID="f281cb1b715a331f578e7486656f3e4efae3f005ebb49297dfba5e53b85b67c5"
Jul 17 10:12:21.218048 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:21.217865    2574 scope.go:117] "RemoveContainer" containerID="68ecb6a1a7856d0bff743fc2ff21d9624ad99203e99774ccb733e9ae3d1ac97b"
Jul 17 10:12:21.218212 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:12:21.218112    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7_odh-ai-gateway-infra(0a858656-ad84-4466-88ad-1da91ab57178)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7" podUID="0a858656-ad84-4466-88ad-1da91ab57178"
Jul 17 10:12:22.226305 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:22.226277    2574 scope.go:117] "RemoveContainer" containerID="68ecb6a1a7856d0bff743fc2ff21d9624ad99203e99774ccb733e9ae3d1ac97b"
Jul 17 10:12:22.226681 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:12:22.226474    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7_odh-ai-gateway-infra(0a858656-ad84-4466-88ad-1da91ab57178)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7" podUID="0a858656-ad84-4466-88ad-1da91ab57178"
Jul 17 10:12:33.180087 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:33.180040    2574 patch_prober.go:28] interesting pod/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.127:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 17 10:12:33.180589 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:33.180117    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl" podUID="8b0de510-d71b-4b55-8217-3c9b8428668b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.127:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 17 10:12:34.298037 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:34.297955    2574 generic.go:356] "Generic (PLEG): container finished" podID="8b0de510-d71b-4b55-8217-3c9b8428668b" containerID="aaa0321094ca1b90a363d9e2f6a4266e151d629cf411dec23c9df9b95bb8f7e0" exitCode=0
Jul 17 10:12:34.298457 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:34.298035    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl" event={"ID":"8b0de510-d71b-4b55-8217-3c9b8428668b","Type":"ContainerDied","Data":"aaa0321094ca1b90a363d9e2f6a4266e151d629cf411dec23c9df9b95bb8f7e0"}
Jul 17 10:12:34.298519 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:34.298504    2574 scope.go:117] "RemoveContainer" containerID="aaa0321094ca1b90a363d9e2f6a4266e151d629cf411dec23c9df9b95bb8f7e0"
Jul 17 10:12:34.299566 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:34.299537    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 10:12:34.299692 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:34.299603    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 10:12:34.299692 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:34.299633    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 10:12:35.309978 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:35.309934    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl" event={"ID":"8b0de510-d71b-4b55-8217-3c9b8428668b","Type":"ContainerStarted","Data":"34436f107a1e0fd19329b72f02df532bc2553cb0244beeb26d5607a14b2da43f"}
Jul 17 10:12:35.510415 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:35.510370    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:35.510824 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:35.510794    2574 patch_prober.go:28] interesting pod/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" start-of-body=
Jul 17 10:12:35.510926 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:35.510860    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl" podUID="8b0de510-d71b-4b55-8217-3c9b8428668b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused"
Jul 17 10:12:36.510975 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:36.510929    2574 patch_prober.go:28] interesting pod/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" start-of-body=
Jul 17 10:12:36.511522 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:36.511023    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl" podUID="8b0de510-d71b-4b55-8217-3c9b8428668b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused"
Jul 17 10:12:37.510056 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:37.510020    2574 patch_prober.go:28] interesting pod/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" start-of-body=
Jul 17 10:12:37.510388 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:37.510356    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl" podUID="8b0de510-d71b-4b55-8217-3c9b8428668b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused"
Jul 17 10:12:37.891607 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:37.891528    2574 scope.go:117] "RemoveContainer" containerID="68ecb6a1a7856d0bff743fc2ff21d9624ad99203e99774ccb733e9ae3d1ac97b"
Jul 17 10:12:38.338129 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:38.338088    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7" event={"ID":"0a858656-ad84-4466-88ad-1da91ab57178","Type":"ContainerStarted","Data":"6c33ab2203f8743e6f6fab0f6f5a78ab958cfa297601991afc47bba57625ec8d"}
Jul 17 10:12:38.358868 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:38.358821    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7" podStartSLOduration=20.358806293 podStartE2EDuration="20.358806293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:12:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:12:38.358637602 +0000 UTC m=+2755.941275977" watchObservedRunningTime="2026-07-17 10:12:38.358806293 +0000 UTC m=+2755.941444742"
Jul 17 10:12:38.510322 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:38.510285    2574 patch_prober.go:28] interesting pod/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" start-of-body=
Jul 17 10:12:38.511023 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:38.510600    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl" podUID="8b0de510-d71b-4b55-8217-3c9b8428668b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused"
Jul 17 10:12:39.345574 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:39.345539    2574 generic.go:356] "Generic (PLEG): container finished" podID="0a858656-ad84-4466-88ad-1da91ab57178" containerID="6c33ab2203f8743e6f6fab0f6f5a78ab958cfa297601991afc47bba57625ec8d" exitCode=6
Jul 17 10:12:39.346005 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:39.345589    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7" event={"ID":"0a858656-ad84-4466-88ad-1da91ab57178","Type":"ContainerDied","Data":"6c33ab2203f8743e6f6fab0f6f5a78ab958cfa297601991afc47bba57625ec8d"}
Jul 17 10:12:39.346005 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:39.345654    2574 scope.go:117] "RemoveContainer" containerID="68ecb6a1a7856d0bff743fc2ff21d9624ad99203e99774ccb733e9ae3d1ac97b"
Jul 17 10:12:39.346005 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:39.345970    2574 scope.go:117] "RemoveContainer" containerID="6c33ab2203f8743e6f6fab0f6f5a78ab958cfa297601991afc47bba57625ec8d"
Jul 17 10:12:39.346267 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:12:39.346215    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7_odh-ai-gateway-infra(0a858656-ad84-4466-88ad-1da91ab57178)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7" podUID="0a858656-ad84-4466-88ad-1da91ab57178"
Jul 17 10:12:39.369808 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:39.369778    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7"]
Jul 17 10:12:39.514036 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:39.514004    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:39.514312 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:39.514288    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:39.514913 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:39.514893    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:12:40.518938 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:40.518912    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7"
Jul 17 10:12:40.557924 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:40.557889    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a858656-ad84-4466-88ad-1da91ab57178-maas-api-service-ca\") pod \"0a858656-ad84-4466-88ad-1da91ab57178\" (UID: \"0a858656-ad84-4466-88ad-1da91ab57178\") "
Jul 17 10:12:40.558323 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:40.558298    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a858656-ad84-4466-88ad-1da91ab57178-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0a858656-ad84-4466-88ad-1da91ab57178" (UID: "0a858656-ad84-4466-88ad-1da91ab57178"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:12:40.658901 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:40.658868    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a858656-ad84-4466-88ad-1da91ab57178-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:12:41.368367 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:41.368332    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7"
Jul 17 10:12:41.368367 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:41.368355    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7" event={"ID":"0a858656-ad84-4466-88ad-1da91ab57178","Type":"ContainerDied","Data":"9cfb35f451608272f7afbcaf5cd5967d6baaedc4f25886645cfef788eb9ae9e2"}
Jul 17 10:12:41.368631 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:41.368431    2574 scope.go:117] "RemoveContainer" containerID="6c33ab2203f8743e6f6fab0f6f5a78ab958cfa297601991afc47bba57625ec8d"
Jul 17 10:12:41.390780 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:41.390753    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7"]
Jul 17 10:12:41.399499 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:41.399476    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-jwtm7"]
Jul 17 10:12:42.898450 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:42.898417    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a858656-ad84-4466-88ad-1da91ab57178" path="/var/lib/kubelet/pods/0a858656-ad84-4466-88ad-1da91ab57178/volumes"
Jul 17 10:12:45.725370 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:45.725330    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-cf4f8"]
Jul 17 10:12:45.725820 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:45.725807    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a858656-ad84-4466-88ad-1da91ab57178" containerName="revoke-keys"
Jul 17 10:12:45.725878 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:45.725822    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a858656-ad84-4466-88ad-1da91ab57178" containerName="revoke-keys"
Jul 17 10:12:45.725878 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:45.725847    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e" containerName="revoke-keys"
Jul 17 10:12:45.725878 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:45.725856    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e" containerName="revoke-keys"
Jul 17 10:12:45.725878 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:45.725869    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e" containerName="revoke-keys"
Jul 17 10:12:45.725878 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:45.725876    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e" containerName="revoke-keys"
Jul 17 10:12:45.726056 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:45.725892    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a858656-ad84-4466-88ad-1da91ab57178" containerName="revoke-keys"
Jul 17 10:12:45.726056 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:45.725897    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a858656-ad84-4466-88ad-1da91ab57178" containerName="revoke-keys"
Jul 17 10:12:45.726056 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:45.725963    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e" containerName="revoke-keys"
Jul 17 10:12:45.726056 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:45.725974    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a858656-ad84-4466-88ad-1da91ab57178" containerName="revoke-keys"
Jul 17 10:12:45.726056 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:45.725982    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a858656-ad84-4466-88ad-1da91ab57178" containerName="revoke-keys"
Jul 17 10:12:45.726056 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:45.725988    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a858656-ad84-4466-88ad-1da91ab57178" containerName="revoke-keys"
Jul 17 10:12:45.726056 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:45.725994    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e" containerName="revoke-keys"
Jul 17 10:12:45.730611 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:45.730594    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-cf4f8"
Jul 17 10:12:45.733730 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:45.733697    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:12:45.736282 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:45.736173    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-cf4f8"]
Jul 17 10:12:45.811104 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:45.811064    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e3214ae-6c34-4d2c-a1a5-3dd40be732ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-cf4f8\" (UID: \"3e3214ae-6c34-4d2c-a1a5-3dd40be732ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-cf4f8"
Jul 17 10:12:45.912076 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:45.912044    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e3214ae-6c34-4d2c-a1a5-3dd40be732ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-cf4f8\" (UID: \"3e3214ae-6c34-4d2c-a1a5-3dd40be732ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-cf4f8"
Jul 17 10:12:45.912702 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:45.912682    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e3214ae-6c34-4d2c-a1a5-3dd40be732ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-cf4f8\" (UID: \"3e3214ae-6c34-4d2c-a1a5-3dd40be732ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-cf4f8"
Jul 17 10:12:46.042792 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:46.042707    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-cf4f8"
Jul 17 10:12:46.181371 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:46.181344    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-cf4f8"]
Jul 17 10:12:46.183387 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:12:46.183354    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e3214ae_6c34_4d2c_a1a5_3dd40be732ca.slice/crio-65aa80baa89b78c42122255fb1b7d8e3fcfef59bb53f8faac1b5d3b5d61b446a WatchSource:0}: Error finding container 65aa80baa89b78c42122255fb1b7d8e3fcfef59bb53f8faac1b5d3b5d61b446a: Status 404 returned error can't find the container with id 65aa80baa89b78c42122255fb1b7d8e3fcfef59bb53f8faac1b5d3b5d61b446a
Jul 17 10:12:46.328760 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:46.328636    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm"]
Jul 17 10:12:46.329100 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:46.329082    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a858656-ad84-4466-88ad-1da91ab57178" containerName="revoke-keys"
Jul 17 10:12:46.329100 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:46.329099    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a858656-ad84-4466-88ad-1da91ab57178" containerName="revoke-keys"
Jul 17 10:12:46.329310 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:46.329131    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e" containerName="revoke-keys"
Jul 17 10:12:46.329310 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:46.329141    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e" containerName="revoke-keys"
Jul 17 10:12:46.329310 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:46.329214    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="b518de5f-3fd4-45a0-8a64-b7ee4bc1c11e" containerName="revoke-keys"
Jul 17 10:12:46.333496 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:46.333478    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm"
Jul 17 10:12:46.340144 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:46.340108    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm"]
Jul 17 10:12:46.400188 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:46.400157    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-cf4f8" event={"ID":"3e3214ae-6c34-4d2c-a1a5-3dd40be732ca","Type":"ContainerStarted","Data":"65aa80baa89b78c42122255fb1b7d8e3fcfef59bb53f8faac1b5d3b5d61b446a"}
Jul 17 10:12:46.417475 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:46.417439    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm\" (UID: \"0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm"
Jul 17 10:12:46.518973 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:46.518947    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm\" (UID: \"0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm"
Jul 17 10:12:46.519547 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:46.519528    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm\" (UID: \"0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm"
Jul 17 10:12:46.646290 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:46.646182    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm"
Jul 17 10:12:46.790274 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:46.790231    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm"]
Jul 17 10:12:46.792170 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:12:46.792142    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cb7cc8d_6f1e_4e75_8f9e_a655cdda8977.slice/crio-9d6af02a943217a22b9e95fa45fc5ca05f68386afa40c2a202f8980e1b8a4ab5 WatchSource:0}: Error finding container 9d6af02a943217a22b9e95fa45fc5ca05f68386afa40c2a202f8980e1b8a4ab5: Status 404 returned error can't find the container with id 9d6af02a943217a22b9e95fa45fc5ca05f68386afa40c2a202f8980e1b8a4ab5
Jul 17 10:12:47.408606 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:47.408567    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm" event={"ID":"0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977","Type":"ContainerStarted","Data":"78dea7b602a4d19cb55508014e369bf183060c0ff8d4178781a89c1c337a85d7"}
Jul 17 10:12:47.408804 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:47.408633    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm" event={"ID":"0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977","Type":"ContainerStarted","Data":"9d6af02a943217a22b9e95fa45fc5ca05f68386afa40c2a202f8980e1b8a4ab5"}
Jul 17 10:12:47.410411 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:47.410379    2574 generic.go:356] "Generic (PLEG): container finished" podID="3e3214ae-6c34-4d2c-a1a5-3dd40be732ca" containerID="fe1becbbe8890dbb017b4c9d1fb8c5e179369cc31b9e508c7e73c2356e4b6ec3" exitCode=6
Jul 17 10:12:47.410541 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:47.410463    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-cf4f8" event={"ID":"3e3214ae-6c34-4d2c-a1a5-3dd40be732ca","Type":"ContainerDied","Data":"fe1becbbe8890dbb017b4c9d1fb8c5e179369cc31b9e508c7e73c2356e4b6ec3"}
Jul 17 10:12:47.410611 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:47.410600    2574 scope.go:117] "RemoveContainer" containerID="fe1becbbe8890dbb017b4c9d1fb8c5e179369cc31b9e508c7e73c2356e4b6ec3"
Jul 17 10:12:47.430430 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:47.430368    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm" podStartSLOduration=1.430352764 podStartE2EDuration="1.430352764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:12:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:12:47.429170507 +0000 UTC m=+2765.011808883" watchObservedRunningTime="2026-07-17 10:12:47.430352764 +0000 UTC m=+2765.012991140"
Jul 17 10:12:48.416753 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:48.416723    2574 generic.go:356] "Generic (PLEG): container finished" podID="0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977" containerID="78dea7b602a4d19cb55508014e369bf183060c0ff8d4178781a89c1c337a85d7" exitCode=6
Jul 17 10:12:48.417258 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:48.416801    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm" event={"ID":"0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977","Type":"ContainerDied","Data":"78dea7b602a4d19cb55508014e369bf183060c0ff8d4178781a89c1c337a85d7"}
Jul 17 10:12:48.417258 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:48.417067    2574 scope.go:117] "RemoveContainer" containerID="78dea7b602a4d19cb55508014e369bf183060c0ff8d4178781a89c1c337a85d7"
Jul 17 10:12:48.418912 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:48.418890    2574 generic.go:356] "Generic (PLEG): container finished" podID="3e3214ae-6c34-4d2c-a1a5-3dd40be732ca" containerID="9927c5c40464749f18484fd3d941e54c5816f57cb8d2cce5a6a03942dcffcc09" exitCode=6
Jul 17 10:12:48.419034 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:48.418960    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-cf4f8" event={"ID":"3e3214ae-6c34-4d2c-a1a5-3dd40be732ca","Type":"ContainerDied","Data":"9927c5c40464749f18484fd3d941e54c5816f57cb8d2cce5a6a03942dcffcc09"}
Jul 17 10:12:48.419034 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:48.419010    2574 scope.go:117] "RemoveContainer" containerID="fe1becbbe8890dbb017b4c9d1fb8c5e179369cc31b9e508c7e73c2356e4b6ec3"
Jul 17 10:12:48.419189 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:48.419171    2574 scope.go:117] "RemoveContainer" containerID="9927c5c40464749f18484fd3d941e54c5816f57cb8d2cce5a6a03942dcffcc09"
Jul 17 10:12:48.419412 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:12:48.419393    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-cf4f8_odh-ai-gateway-infra(3e3214ae-6c34-4d2c-a1a5-3dd40be732ca)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-cf4f8" podUID="3e3214ae-6c34-4d2c-a1a5-3dd40be732ca"
Jul 17 10:12:49.427488 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:49.427450    2574 generic.go:356] "Generic (PLEG): container finished" podID="0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977" containerID="c1ea21a196eb161d5b3707abd5269a153a1e52c963c1e06428798247c93e0ba0" exitCode=6
Jul 17 10:12:49.427957 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:49.427531    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm" event={"ID":"0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977","Type":"ContainerDied","Data":"c1ea21a196eb161d5b3707abd5269a153a1e52c963c1e06428798247c93e0ba0"}
Jul 17 10:12:49.427957 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:49.427607    2574 scope.go:117] "RemoveContainer" containerID="78dea7b602a4d19cb55508014e369bf183060c0ff8d4178781a89c1c337a85d7"
Jul 17 10:12:49.427957 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:49.427869    2574 scope.go:117] "RemoveContainer" containerID="c1ea21a196eb161d5b3707abd5269a153a1e52c963c1e06428798247c93e0ba0"
Jul 17 10:12:49.428167 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:12:49.428144    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm_odh-ai-gateway-infra(0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm" podUID="0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977"
Jul 17 10:12:49.429530 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:49.429513    2574 scope.go:117] "RemoveContainer" containerID="9927c5c40464749f18484fd3d941e54c5816f57cb8d2cce5a6a03942dcffcc09"
Jul 17 10:12:49.429726 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:12:49.429709    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-cf4f8_odh-ai-gateway-infra(3e3214ae-6c34-4d2c-a1a5-3dd40be732ca)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-cf4f8" podUID="3e3214ae-6c34-4d2c-a1a5-3dd40be732ca"
Jul 17 10:12:50.436918 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:50.436892    2574 scope.go:117] "RemoveContainer" containerID="c1ea21a196eb161d5b3707abd5269a153a1e52c963c1e06428798247c93e0ba0"
Jul 17 10:12:50.437358 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:12:50.437135    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm_odh-ai-gateway-infra(0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm" podUID="0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977"
Jul 17 10:12:57.357300 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:57.357268    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6"]
Jul 17 10:12:57.360805 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:57.360787    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6"
Jul 17 10:12:57.368326 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:57.368294    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6"]
Jul 17 10:12:57.431500 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:57.431461    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f16d6aca-2c6c-4e7a-8382-d91851e626bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6\" (UID: \"f16d6aca-2c6c-4e7a-8382-d91851e626bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6"
Jul 17 10:12:57.532151 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:57.532112    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f16d6aca-2c6c-4e7a-8382-d91851e626bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6\" (UID: \"f16d6aca-2c6c-4e7a-8382-d91851e626bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6"
Jul 17 10:12:57.532798 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:57.532776    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f16d6aca-2c6c-4e7a-8382-d91851e626bc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6\" (UID: \"f16d6aca-2c6c-4e7a-8382-d91851e626bc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6"
Jul 17 10:12:57.673921 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:57.673882    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6"
Jul 17 10:12:58.025407 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:58.025383    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6"]
Jul 17 10:12:58.027447 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:12:58.027422    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf16d6aca_2c6c_4e7a_8382_d91851e626bc.slice/crio-d6dceca3409643b2730865cc3142827097210efe9c8ba8c93fb1250545e89700 WatchSource:0}: Error finding container d6dceca3409643b2730865cc3142827097210efe9c8ba8c93fb1250545e89700: Status 404 returned error can't find the container with id d6dceca3409643b2730865cc3142827097210efe9c8ba8c93fb1250545e89700
Jul 17 10:12:58.480741 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:58.480702    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6" event={"ID":"f16d6aca-2c6c-4e7a-8382-d91851e626bc","Type":"ContainerStarted","Data":"52c1c6f5b2eae302cd1fa6928d97b6f8108627720e4137c6ef83fab7fdacfbb8"}
Jul 17 10:12:58.481127 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:58.480770    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6" event={"ID":"f16d6aca-2c6c-4e7a-8382-d91851e626bc","Type":"ContainerStarted","Data":"d6dceca3409643b2730865cc3142827097210efe9c8ba8c93fb1250545e89700"}
Jul 17 10:12:58.498255 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:58.498185    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6" podStartSLOduration=1.498169425 podStartE2EDuration="1.498169425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:12:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:12:58.496878759 +0000 UTC m=+2776.079517147" watchObservedRunningTime="2026-07-17 10:12:58.498169425 +0000 UTC m=+2776.080807800"
Jul 17 10:12:59.488694 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:59.488656    2574 generic.go:356] "Generic (PLEG): container finished" podID="f16d6aca-2c6c-4e7a-8382-d91851e626bc" containerID="52c1c6f5b2eae302cd1fa6928d97b6f8108627720e4137c6ef83fab7fdacfbb8" exitCode=6
Jul 17 10:12:59.489192 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:59.488742    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6" event={"ID":"f16d6aca-2c6c-4e7a-8382-d91851e626bc","Type":"ContainerDied","Data":"52c1c6f5b2eae302cd1fa6928d97b6f8108627720e4137c6ef83fab7fdacfbb8"}
Jul 17 10:12:59.489192 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:12:59.488933    2574 scope.go:117] "RemoveContainer" containerID="52c1c6f5b2eae302cd1fa6928d97b6f8108627720e4137c6ef83fab7fdacfbb8"
Jul 17 10:13:00.496035 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:00.495992    2574 generic.go:356] "Generic (PLEG): container finished" podID="f16d6aca-2c6c-4e7a-8382-d91851e626bc" containerID="655d4888fc9237ee8c5c09fa573d84c2a9929919bf7fcebd77363d41c37523ee" exitCode=6
Jul 17 10:13:00.496492 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:00.496081    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6" event={"ID":"f16d6aca-2c6c-4e7a-8382-d91851e626bc","Type":"ContainerDied","Data":"655d4888fc9237ee8c5c09fa573d84c2a9929919bf7fcebd77363d41c37523ee"}
Jul 17 10:13:00.496492 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:00.496139    2574 scope.go:117] "RemoveContainer" containerID="52c1c6f5b2eae302cd1fa6928d97b6f8108627720e4137c6ef83fab7fdacfbb8"
Jul 17 10:13:00.496492 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:00.496394    2574 scope.go:117] "RemoveContainer" containerID="655d4888fc9237ee8c5c09fa573d84c2a9929919bf7fcebd77363d41c37523ee"
Jul 17 10:13:00.496661 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:13:00.496639    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6_odh-ai-gateway-infra(f16d6aca-2c6c-4e7a-8382-d91851e626bc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6" podUID="f16d6aca-2c6c-4e7a-8382-d91851e626bc"
Jul 17 10:13:00.892321 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:00.892209    2574 scope.go:117] "RemoveContainer" containerID="9927c5c40464749f18484fd3d941e54c5816f57cb8d2cce5a6a03942dcffcc09"
Jul 17 10:13:01.504735 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:01.504641    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-cf4f8" event={"ID":"3e3214ae-6c34-4d2c-a1a5-3dd40be732ca","Type":"ContainerStarted","Data":"a119bca84d9a9ac8b8acc3e1b464a11515def6fcd5a03f032a6525709843cc3d"}
Jul 17 10:13:01.506954 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:01.506936    2574 scope.go:117] "RemoveContainer" containerID="655d4888fc9237ee8c5c09fa573d84c2a9929919bf7fcebd77363d41c37523ee"
Jul 17 10:13:01.507159 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:13:01.507139    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6_odh-ai-gateway-infra(f16d6aca-2c6c-4e7a-8382-d91851e626bc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6" podUID="f16d6aca-2c6c-4e7a-8382-d91851e626bc"
Jul 17 10:13:01.522120 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:01.522076    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-cf4f8" podStartSLOduration=16.522063222 podStartE2EDuration="16.522063222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:12:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:13:01.520594174 +0000 UTC m=+2779.103232550" watchObservedRunningTime="2026-07-17 10:13:01.522063222 +0000 UTC m=+2779.104701638"
Jul 17 10:13:02.514664 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:02.514631    2574 generic.go:356] "Generic (PLEG): container finished" podID="3e3214ae-6c34-4d2c-a1a5-3dd40be732ca" containerID="a119bca84d9a9ac8b8acc3e1b464a11515def6fcd5a03f032a6525709843cc3d" exitCode=6
Jul 17 10:13:02.515142 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:02.514712    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-cf4f8" event={"ID":"3e3214ae-6c34-4d2c-a1a5-3dd40be732ca","Type":"ContainerDied","Data":"a119bca84d9a9ac8b8acc3e1b464a11515def6fcd5a03f032a6525709843cc3d"}
Jul 17 10:13:02.515142 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:02.514771    2574 scope.go:117] "RemoveContainer" containerID="9927c5c40464749f18484fd3d941e54c5816f57cb8d2cce5a6a03942dcffcc09"
Jul 17 10:13:02.515142 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:02.515054    2574 scope.go:117] "RemoveContainer" containerID="a119bca84d9a9ac8b8acc3e1b464a11515def6fcd5a03f032a6525709843cc3d"
Jul 17 10:13:02.515350 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:13:02.515322    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-cf4f8_odh-ai-gateway-infra(3e3214ae-6c34-4d2c-a1a5-3dd40be732ca)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-cf4f8" podUID="3e3214ae-6c34-4d2c-a1a5-3dd40be732ca"
Jul 17 10:13:02.531890 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:02.531859    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-cf4f8"]
Jul 17 10:13:03.677395 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:03.677371    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-cf4f8"
Jul 17 10:13:03.798560 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:03.798470    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e3214ae-6c34-4d2c-a1a5-3dd40be732ca-maas-api-service-ca\") pod \"3e3214ae-6c34-4d2c-a1a5-3dd40be732ca\" (UID: \"3e3214ae-6c34-4d2c-a1a5-3dd40be732ca\") "
Jul 17 10:13:03.798835 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:03.798809    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e3214ae-6c34-4d2c-a1a5-3dd40be732ca-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3e3214ae-6c34-4d2c-a1a5-3dd40be732ca" (UID: "3e3214ae-6c34-4d2c-a1a5-3dd40be732ca"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:13:03.900109 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:03.900077    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3e3214ae-6c34-4d2c-a1a5-3dd40be732ca-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:13:04.529166 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:04.529130    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-cf4f8" event={"ID":"3e3214ae-6c34-4d2c-a1a5-3dd40be732ca","Type":"ContainerDied","Data":"65aa80baa89b78c42122255fb1b7d8e3fcfef59bb53f8faac1b5d3b5d61b446a"}
Jul 17 10:13:04.529166 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:04.529168    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-cf4f8"
Jul 17 10:13:04.529445 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:04.529201    2574 scope.go:117] "RemoveContainer" containerID="a119bca84d9a9ac8b8acc3e1b464a11515def6fcd5a03f032a6525709843cc3d"
Jul 17 10:13:04.551122 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:04.551094    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-cf4f8"]
Jul 17 10:13:04.559734 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:04.559708    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-cf4f8"]
Jul 17 10:13:04.891886 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:04.891795    2574 scope.go:117] "RemoveContainer" containerID="c1ea21a196eb161d5b3707abd5269a153a1e52c963c1e06428798247c93e0ba0"
Jul 17 10:13:04.898578 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:04.898553    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e3214ae-6c34-4d2c-a1a5-3dd40be732ca" path="/var/lib/kubelet/pods/3e3214ae-6c34-4d2c-a1a5-3dd40be732ca/volumes"
Jul 17 10:13:05.536682 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:05.536648    2574 generic.go:356] "Generic (PLEG): container finished" podID="0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977" containerID="b15b6f2289c23fb1a04ba9f2ec5441b4fa5ae144f08d6fda35fd1f5ba07cd21a" exitCode=6
Jul 17 10:13:05.536870 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:05.536722    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm" event={"ID":"0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977","Type":"ContainerDied","Data":"b15b6f2289c23fb1a04ba9f2ec5441b4fa5ae144f08d6fda35fd1f5ba07cd21a"}
Jul 17 10:13:05.536870 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:05.536789    2574 scope.go:117] "RemoveContainer" containerID="c1ea21a196eb161d5b3707abd5269a153a1e52c963c1e06428798247c93e0ba0"
Jul 17 10:13:05.537136 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:05.537116    2574 scope.go:117] "RemoveContainer" containerID="b15b6f2289c23fb1a04ba9f2ec5441b4fa5ae144f08d6fda35fd1f5ba07cd21a"
Jul 17 10:13:05.537419 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:13:05.537400    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm_odh-ai-gateway-infra(0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm" podUID="0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977"
Jul 17 10:13:06.563254 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:06.563198    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm"]
Jul 17 10:13:06.722329 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:06.722305    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm"
Jul 17 10:13:06.827893 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:06.827809    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977-maas-api-service-ca\") pod \"0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977\" (UID: \"0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977\") "
Jul 17 10:13:06.828164 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:06.828142    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977" (UID: "0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:13:06.929626 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:06.929594    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:13:07.551904 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:07.551869    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm" event={"ID":"0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977","Type":"ContainerDied","Data":"9d6af02a943217a22b9e95fa45fc5ca05f68386afa40c2a202f8980e1b8a4ab5"}
Jul 17 10:13:07.552085 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:07.551882    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm"
Jul 17 10:13:07.552085 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:07.551960    2574 scope.go:117] "RemoveContainer" containerID="b15b6f2289c23fb1a04ba9f2ec5441b4fa5ae144f08d6fda35fd1f5ba07cd21a"
Jul 17 10:13:07.570082 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:07.570056    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm"]
Jul 17 10:13:07.575749 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:07.575726    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-p29hm"]
Jul 17 10:13:08.898520 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:08.898488    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977" path="/var/lib/kubelet/pods/0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977/volumes"
Jul 17 10:13:12.894740 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:12.894714    2574 scope.go:117] "RemoveContainer" containerID="655d4888fc9237ee8c5c09fa573d84c2a9929919bf7fcebd77363d41c37523ee"
Jul 17 10:13:13.587673 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:13.587639    2574 generic.go:356] "Generic (PLEG): container finished" podID="f16d6aca-2c6c-4e7a-8382-d91851e626bc" containerID="88163cf7c5ea9b63eb3ba861e9edac33887a02eef49432cce850fdbc4ab192a7" exitCode=6
Jul 17 10:13:13.587835 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:13.587703    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6" event={"ID":"f16d6aca-2c6c-4e7a-8382-d91851e626bc","Type":"ContainerDied","Data":"88163cf7c5ea9b63eb3ba861e9edac33887a02eef49432cce850fdbc4ab192a7"}
Jul 17 10:13:13.587835 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:13.587775    2574 scope.go:117] "RemoveContainer" containerID="655d4888fc9237ee8c5c09fa573d84c2a9929919bf7fcebd77363d41c37523ee"
Jul 17 10:13:13.588115 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:13.588092    2574 scope.go:117] "RemoveContainer" containerID="88163cf7c5ea9b63eb3ba861e9edac33887a02eef49432cce850fdbc4ab192a7"
Jul 17 10:13:13.588383 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:13:13.588357    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6_odh-ai-gateway-infra(f16d6aca-2c6c-4e7a-8382-d91851e626bc)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6" podUID="f16d6aca-2c6c-4e7a-8382-d91851e626bc"
Jul 17 10:13:14.620160 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:14.617786    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6"]
Jul 17 10:13:14.772820 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:14.772794    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6"
Jul 17 10:13:14.909941 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:14.909920    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f16d6aca-2c6c-4e7a-8382-d91851e626bc-maas-api-service-ca\") pod \"f16d6aca-2c6c-4e7a-8382-d91851e626bc\" (UID: \"f16d6aca-2c6c-4e7a-8382-d91851e626bc\") "
Jul 17 10:13:14.910299 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:14.910275    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f16d6aca-2c6c-4e7a-8382-d91851e626bc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f16d6aca-2c6c-4e7a-8382-d91851e626bc" (UID: "f16d6aca-2c6c-4e7a-8382-d91851e626bc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:13:15.011696 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:15.011659    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f16d6aca-2c6c-4e7a-8382-d91851e626bc-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:13:15.604253 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:15.604205    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6" event={"ID":"f16d6aca-2c6c-4e7a-8382-d91851e626bc","Type":"ContainerDied","Data":"d6dceca3409643b2730865cc3142827097210efe9c8ba8c93fb1250545e89700"}
Jul 17 10:13:15.604437 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:15.604286    2574 scope.go:117] "RemoveContainer" containerID="88163cf7c5ea9b63eb3ba861e9edac33887a02eef49432cce850fdbc4ab192a7"
Jul 17 10:13:15.604437 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:15.604215    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6"
Jul 17 10:13:15.630996 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:15.630971    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6"]
Jul 17 10:13:15.633511 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:15.633491    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2qgs6"]
Jul 17 10:13:16.898571 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:16.898536    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f16d6aca-2c6c-4e7a-8382-d91851e626bc" path="/var/lib/kubelet/pods/f16d6aca-2c6c-4e7a-8382-d91851e626bc/volumes"
Jul 17 10:13:18.508985 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:18.508951    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-4p96b"]
Jul 17 10:13:18.509601 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:18.509585    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f16d6aca-2c6c-4e7a-8382-d91851e626bc" containerName="revoke-keys"
Jul 17 10:13:18.509647 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:18.509604    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f16d6aca-2c6c-4e7a-8382-d91851e626bc" containerName="revoke-keys"
Jul 17 10:13:18.509647 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:18.509620    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977" containerName="revoke-keys"
Jul 17 10:13:18.509647 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:18.509629    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977" containerName="revoke-keys"
Jul 17 10:13:18.509647 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:18.509641    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e3214ae-6c34-4d2c-a1a5-3dd40be732ca" containerName="revoke-keys"
Jul 17 10:13:18.509791 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:18.509650    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e3214ae-6c34-4d2c-a1a5-3dd40be732ca" containerName="revoke-keys"
Jul 17 10:13:18.509791 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:18.509662    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977" containerName="revoke-keys"
Jul 17 10:13:18.509791 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:18.509670    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977" containerName="revoke-keys"
Jul 17 10:13:18.509791 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:18.509680    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f16d6aca-2c6c-4e7a-8382-d91851e626bc" containerName="revoke-keys"
Jul 17 10:13:18.509791 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:18.509688    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f16d6aca-2c6c-4e7a-8382-d91851e626bc" containerName="revoke-keys"
Jul 17 10:13:18.509791 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:18.509728    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e3214ae-6c34-4d2c-a1a5-3dd40be732ca" containerName="revoke-keys"
Jul 17 10:13:18.509791 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:18.509735    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e3214ae-6c34-4d2c-a1a5-3dd40be732ca" containerName="revoke-keys"
Jul 17 10:13:18.510006 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:18.509826    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977" containerName="revoke-keys"
Jul 17 10:13:18.510006 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:18.509837    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e3214ae-6c34-4d2c-a1a5-3dd40be732ca" containerName="revoke-keys"
Jul 17 10:13:18.510006 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:18.509847    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f16d6aca-2c6c-4e7a-8382-d91851e626bc" containerName="revoke-keys"
Jul 17 10:13:18.510006 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:18.509860    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977" containerName="revoke-keys"
Jul 17 10:13:18.510006 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:18.509869    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f16d6aca-2c6c-4e7a-8382-d91851e626bc" containerName="revoke-keys"
Jul 17 10:13:18.510006 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:18.509879    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e3214ae-6c34-4d2c-a1a5-3dd40be732ca" containerName="revoke-keys"
Jul 17 10:13:18.514872 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:18.514855    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-4p96b"
Jul 17 10:13:18.518910 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:18.518885    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:13:18.521076 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:18.521048    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-4p96b"]
Jul 17 10:13:18.540413 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:18.540388    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed3c6f58-5662-46ec-9420-b8d927cb5290-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-4p96b\" (UID: \"ed3c6f58-5662-46ec-9420-b8d927cb5290\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-4p96b"
Jul 17 10:13:18.641193 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:18.641159    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed3c6f58-5662-46ec-9420-b8d927cb5290-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-4p96b\" (UID: \"ed3c6f58-5662-46ec-9420-b8d927cb5290\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-4p96b"
Jul 17 10:13:18.641862 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:18.641841    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed3c6f58-5662-46ec-9420-b8d927cb5290-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-4p96b\" (UID: \"ed3c6f58-5662-46ec-9420-b8d927cb5290\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-4p96b"
Jul 17 10:13:18.827932 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:18.827831    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-4p96b"
Jul 17 10:13:19.176018 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:19.175993    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-4p96b"]
Jul 17 10:13:19.177803 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:13:19.177766    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded3c6f58_5662_46ec_9420_b8d927cb5290.slice/crio-7389f94dab35d9a4afb7039ead3bcdc392497b07f310c9e350c4d7f4e23d2cd0 WatchSource:0}: Error finding container 7389f94dab35d9a4afb7039ead3bcdc392497b07f310c9e350c4d7f4e23d2cd0: Status 404 returned error can't find the container with id 7389f94dab35d9a4afb7039ead3bcdc392497b07f310c9e350c4d7f4e23d2cd0
Jul 17 10:13:19.636491 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:19.636440    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-4p96b" event={"ID":"ed3c6f58-5662-46ec-9420-b8d927cb5290","Type":"ContainerStarted","Data":"2ce9ea4f82336361e94b9ee906cd0ae16f10b03e330fd490b4d4119ee968029d"}
Jul 17 10:13:19.636946 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:19.636542    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-4p96b" event={"ID":"ed3c6f58-5662-46ec-9420-b8d927cb5290","Type":"ContainerStarted","Data":"7389f94dab35d9a4afb7039ead3bcdc392497b07f310c9e350c4d7f4e23d2cd0"}
Jul 17 10:13:19.653648 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:19.653594    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-4p96b" podStartSLOduration=1.6535786780000001 podStartE2EDuration="1.653578678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:13:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:13:19.651341167 +0000 UTC m=+2797.233979541" watchObservedRunningTime="2026-07-17 10:13:19.653578678 +0000 UTC m=+2797.236217054"
Jul 17 10:13:20.643406 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:20.643373    2574 generic.go:356] "Generic (PLEG): container finished" podID="ed3c6f58-5662-46ec-9420-b8d927cb5290" containerID="2ce9ea4f82336361e94b9ee906cd0ae16f10b03e330fd490b4d4119ee968029d" exitCode=6
Jul 17 10:13:20.643790 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:20.643430    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-4p96b" event={"ID":"ed3c6f58-5662-46ec-9420-b8d927cb5290","Type":"ContainerDied","Data":"2ce9ea4f82336361e94b9ee906cd0ae16f10b03e330fd490b4d4119ee968029d"}
Jul 17 10:13:20.643790 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:20.643724    2574 scope.go:117] "RemoveContainer" containerID="2ce9ea4f82336361e94b9ee906cd0ae16f10b03e330fd490b4d4119ee968029d"
Jul 17 10:13:21.651683 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:21.651584    2574 generic.go:356] "Generic (PLEG): container finished" podID="ed3c6f58-5662-46ec-9420-b8d927cb5290" containerID="ece5ca1acd36f16c3214c23c244d8b7759a7ff0fb00eefe908531217f91818fb" exitCode=6
Jul 17 10:13:21.651683 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:21.651626    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-4p96b" event={"ID":"ed3c6f58-5662-46ec-9420-b8d927cb5290","Type":"ContainerDied","Data":"ece5ca1acd36f16c3214c23c244d8b7759a7ff0fb00eefe908531217f91818fb"}
Jul 17 10:13:21.652139 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:21.651700    2574 scope.go:117] "RemoveContainer" containerID="2ce9ea4f82336361e94b9ee906cd0ae16f10b03e330fd490b4d4119ee968029d"
Jul 17 10:13:21.652139 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:21.651916    2574 scope.go:117] "RemoveContainer" containerID="ece5ca1acd36f16c3214c23c244d8b7759a7ff0fb00eefe908531217f91818fb"
Jul 17 10:13:21.652219 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:13:21.652142    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-4p96b_odh-ai-gateway-infra(ed3c6f58-5662-46ec-9420-b8d927cb5290)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-4p96b" podUID="ed3c6f58-5662-46ec-9420-b8d927cb5290"
Jul 17 10:13:22.659168 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:22.659140    2574 scope.go:117] "RemoveContainer" containerID="ece5ca1acd36f16c3214c23c244d8b7759a7ff0fb00eefe908531217f91818fb"
Jul 17 10:13:22.659578 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:13:22.659398    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-4p96b_odh-ai-gateway-infra(ed3c6f58-5662-46ec-9420-b8d927cb5290)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-4p96b" podUID="ed3c6f58-5662-46ec-9420-b8d927cb5290"
Jul 17 10:13:35.763073 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:35.763034    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-ljz26"]
Jul 17 10:13:35.763743 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:35.763723    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e3214ae-6c34-4d2c-a1a5-3dd40be732ca" containerName="revoke-keys"
Jul 17 10:13:35.763789 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:35.763748    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e3214ae-6c34-4d2c-a1a5-3dd40be732ca" containerName="revoke-keys"
Jul 17 10:13:35.763789 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:35.763757    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977" containerName="revoke-keys"
Jul 17 10:13:35.763789 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:35.763762    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977" containerName="revoke-keys"
Jul 17 10:13:35.763789 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:35.763769    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f16d6aca-2c6c-4e7a-8382-d91851e626bc" containerName="revoke-keys"
Jul 17 10:13:35.763789 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:35.763774    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="f16d6aca-2c6c-4e7a-8382-d91851e626bc" containerName="revoke-keys"
Jul 17 10:13:35.763989 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:35.763841    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e3214ae-6c34-4d2c-a1a5-3dd40be732ca" containerName="revoke-keys"
Jul 17 10:13:35.763989 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:35.763850    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cb7cc8d-6f1e-4e75-8f9e-a655cdda8977" containerName="revoke-keys"
Jul 17 10:13:35.763989 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:35.763857    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="f16d6aca-2c6c-4e7a-8382-d91851e626bc" containerName="revoke-keys"
Jul 17 10:13:35.768456 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:35.768436    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-ljz26"
Jul 17 10:13:35.773933 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:35.773898    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-ljz26"]
Jul 17 10:13:35.804417 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:35.804376    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/299b9375-bc51-452a-b3ad-fd1f659454ef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-ljz26\" (UID: \"299b9375-bc51-452a-b3ad-fd1f659454ef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-ljz26"
Jul 17 10:13:35.905670 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:35.905633    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/299b9375-bc51-452a-b3ad-fd1f659454ef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-ljz26\" (UID: \"299b9375-bc51-452a-b3ad-fd1f659454ef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-ljz26"
Jul 17 10:13:35.906421 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:35.906395    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/299b9375-bc51-452a-b3ad-fd1f659454ef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-ljz26\" (UID: \"299b9375-bc51-452a-b3ad-fd1f659454ef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-ljz26"
Jul 17 10:13:36.082224 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:36.082132    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-ljz26"
Jul 17 10:13:36.442251 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:36.442209    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-ljz26"]
Jul 17 10:13:36.445053 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:13:36.445014    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod299b9375_bc51_452a_b3ad_fd1f659454ef.slice/crio-ace0cafda25fba4957d63aa7e8edf9bea151269e07d3c2589963b54a41dd9f14 WatchSource:0}: Error finding container ace0cafda25fba4957d63aa7e8edf9bea151269e07d3c2589963b54a41dd9f14: Status 404 returned error can't find the container with id ace0cafda25fba4957d63aa7e8edf9bea151269e07d3c2589963b54a41dd9f14
Jul 17 10:13:36.740304 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:36.740264    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-ljz26" event={"ID":"299b9375-bc51-452a-b3ad-fd1f659454ef","Type":"ContainerStarted","Data":"8aa4107212883a18e9c8533d86a8a44cda291150f21d09e59e398c3adad0a0e4"}
Jul 17 10:13:36.740443 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:36.740327    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-ljz26" event={"ID":"299b9375-bc51-452a-b3ad-fd1f659454ef","Type":"ContainerStarted","Data":"ace0cafda25fba4957d63aa7e8edf9bea151269e07d3c2589963b54a41dd9f14"}
Jul 17 10:13:36.759136 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:36.759082    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-ljz26" podStartSLOduration=1.7590666019999999 podStartE2EDuration="1.759066602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:13:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:13:36.756983201 +0000 UTC m=+2814.339621596" watchObservedRunningTime="2026-07-17 10:13:36.759066602 +0000 UTC m=+2814.341704978"
Jul 17 10:13:36.898366 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:36.898337    2574 scope.go:117] "RemoveContainer" containerID="ece5ca1acd36f16c3214c23c244d8b7759a7ff0fb00eefe908531217f91818fb"
Jul 17 10:13:37.747426 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:37.747393    2574 generic.go:356] "Generic (PLEG): container finished" podID="ed3c6f58-5662-46ec-9420-b8d927cb5290" containerID="61b443c887da16db6cb963aa18f667db0ec138482f372edad20878d1cba10834" exitCode=6
Jul 17 10:13:37.747624 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:37.747471    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-4p96b" event={"ID":"ed3c6f58-5662-46ec-9420-b8d927cb5290","Type":"ContainerDied","Data":"61b443c887da16db6cb963aa18f667db0ec138482f372edad20878d1cba10834"}
Jul 17 10:13:37.747624 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:37.747530    2574 scope.go:117] "RemoveContainer" containerID="ece5ca1acd36f16c3214c23c244d8b7759a7ff0fb00eefe908531217f91818fb"
Jul 17 10:13:37.747839 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:37.747822    2574 scope.go:117] "RemoveContainer" containerID="61b443c887da16db6cb963aa18f667db0ec138482f372edad20878d1cba10834"
Jul 17 10:13:37.748145 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:13:37.748108    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-4p96b_odh-ai-gateway-infra(ed3c6f58-5662-46ec-9420-b8d927cb5290)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-4p96b" podUID="ed3c6f58-5662-46ec-9420-b8d927cb5290"
Jul 17 10:13:37.749339 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:37.749312    2574 generic.go:356] "Generic (PLEG): container finished" podID="299b9375-bc51-452a-b3ad-fd1f659454ef" containerID="8aa4107212883a18e9c8533d86a8a44cda291150f21d09e59e398c3adad0a0e4" exitCode=6
Jul 17 10:13:37.749447 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:37.749356    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-ljz26" event={"ID":"299b9375-bc51-452a-b3ad-fd1f659454ef","Type":"ContainerDied","Data":"8aa4107212883a18e9c8533d86a8a44cda291150f21d09e59e398c3adad0a0e4"}
Jul 17 10:13:37.749603 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:37.749588    2574 scope.go:117] "RemoveContainer" containerID="8aa4107212883a18e9c8533d86a8a44cda291150f21d09e59e398c3adad0a0e4"
Jul 17 10:13:38.756776 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:38.756741    2574 generic.go:356] "Generic (PLEG): container finished" podID="299b9375-bc51-452a-b3ad-fd1f659454ef" containerID="4d9e26b6552cabcccccb1e422d67588129a75921175595b73eae44d4b09bf662" exitCode=6
Jul 17 10:13:38.757228 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:38.756819    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-ljz26" event={"ID":"299b9375-bc51-452a-b3ad-fd1f659454ef","Type":"ContainerDied","Data":"4d9e26b6552cabcccccb1e422d67588129a75921175595b73eae44d4b09bf662"}
Jul 17 10:13:38.757228 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:38.756902    2574 scope.go:117] "RemoveContainer" containerID="8aa4107212883a18e9c8533d86a8a44cda291150f21d09e59e398c3adad0a0e4"
Jul 17 10:13:38.757228 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:38.757127    2574 scope.go:117] "RemoveContainer" containerID="4d9e26b6552cabcccccb1e422d67588129a75921175595b73eae44d4b09bf662"
Jul 17 10:13:38.757437 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:13:38.757411    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-ljz26_odh-ai-gateway-infra(299b9375-bc51-452a-b3ad-fd1f659454ef)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-ljz26" podUID="299b9375-bc51-452a-b3ad-fd1f659454ef"
Jul 17 10:13:38.781748 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:38.781720    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-4p96b"]
Jul 17 10:13:38.937650 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:38.937618    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-4p96b"
Jul 17 10:13:39.040576 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:39.040482    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed3c6f58-5662-46ec-9420-b8d927cb5290-maas-api-service-ca\") pod \"ed3c6f58-5662-46ec-9420-b8d927cb5290\" (UID: \"ed3c6f58-5662-46ec-9420-b8d927cb5290\") "
Jul 17 10:13:39.040807 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:39.040785    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed3c6f58-5662-46ec-9420-b8d927cb5290-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ed3c6f58-5662-46ec-9420-b8d927cb5290" (UID: "ed3c6f58-5662-46ec-9420-b8d927cb5290"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:13:39.141743 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:39.141702    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed3c6f58-5662-46ec-9420-b8d927cb5290-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:13:39.766731 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:39.766704    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-4p96b"
Jul 17 10:13:39.767192 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:39.766741    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-4p96b" event={"ID":"ed3c6f58-5662-46ec-9420-b8d927cb5290","Type":"ContainerDied","Data":"7389f94dab35d9a4afb7039ead3bcdc392497b07f310c9e350c4d7f4e23d2cd0"}
Jul 17 10:13:39.767192 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:39.766819    2574 scope.go:117] "RemoveContainer" containerID="61b443c887da16db6cb963aa18f667db0ec138482f372edad20878d1cba10834"
Jul 17 10:13:39.768797 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:39.768777    2574 scope.go:117] "RemoveContainer" containerID="4d9e26b6552cabcccccb1e422d67588129a75921175595b73eae44d4b09bf662"
Jul 17 10:13:39.769012 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:13:39.768992    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-ljz26_odh-ai-gateway-infra(299b9375-bc51-452a-b3ad-fd1f659454ef)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-ljz26" podUID="299b9375-bc51-452a-b3ad-fd1f659454ef"
Jul 17 10:13:39.805428 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:39.805393    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-4p96b"]
Jul 17 10:13:39.807583 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:39.807563    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-4p96b"]
Jul 17 10:13:40.898462 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:40.898426    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed3c6f58-5662-46ec-9420-b8d927cb5290" path="/var/lib/kubelet/pods/ed3c6f58-5662-46ec-9420-b8d927cb5290/volumes"
Jul 17 10:13:46.377921 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:46.377876    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tfdld"]
Jul 17 10:13:46.378581 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:46.378559    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed3c6f58-5662-46ec-9420-b8d927cb5290" containerName="revoke-keys"
Jul 17 10:13:46.378673 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:46.378585    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed3c6f58-5662-46ec-9420-b8d927cb5290" containerName="revoke-keys"
Jul 17 10:13:46.378673 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:46.378605    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed3c6f58-5662-46ec-9420-b8d927cb5290" containerName="revoke-keys"
Jul 17 10:13:46.378673 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:46.378613    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed3c6f58-5662-46ec-9420-b8d927cb5290" containerName="revoke-keys"
Jul 17 10:13:46.378673 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:46.378639    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed3c6f58-5662-46ec-9420-b8d927cb5290" containerName="revoke-keys"
Jul 17 10:13:46.378673 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:46.378648    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed3c6f58-5662-46ec-9420-b8d927cb5290" containerName="revoke-keys"
Jul 17 10:13:46.378938 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:46.378757    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed3c6f58-5662-46ec-9420-b8d927cb5290" containerName="revoke-keys"
Jul 17 10:13:46.378938 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:46.378772    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed3c6f58-5662-46ec-9420-b8d927cb5290" containerName="revoke-keys"
Jul 17 10:13:46.383159 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:46.383136    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tfdld"
Jul 17 10:13:46.388766 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:46.388734    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tfdld"]
Jul 17 10:13:46.524379 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:46.524333    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85b50483-a861-4c9c-a8e3-4b66857e2a04-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-tfdld\" (UID: \"85b50483-a861-4c9c-a8e3-4b66857e2a04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tfdld"
Jul 17 10:13:46.625807 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:46.625765    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85b50483-a861-4c9c-a8e3-4b66857e2a04-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-tfdld\" (UID: \"85b50483-a861-4c9c-a8e3-4b66857e2a04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tfdld"
Jul 17 10:13:46.626427 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:46.626409    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85b50483-a861-4c9c-a8e3-4b66857e2a04-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-tfdld\" (UID: \"85b50483-a861-4c9c-a8e3-4b66857e2a04\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tfdld"
Jul 17 10:13:46.696911 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:46.696885    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tfdld"
Jul 17 10:13:46.844084 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:46.844059    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tfdld"]
Jul 17 10:13:46.844900 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:13:46.844878    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85b50483_a861_4c9c_a8e3_4b66857e2a04.slice/crio-78a5758d8d0a2e136d05f05a724c7b275a26e23f71ecf9e08baacca437117083 WatchSource:0}: Error finding container 78a5758d8d0a2e136d05f05a724c7b275a26e23f71ecf9e08baacca437117083: Status 404 returned error can't find the container with id 78a5758d8d0a2e136d05f05a724c7b275a26e23f71ecf9e08baacca437117083
Jul 17 10:13:47.402099 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:47.402005    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-v44mr"]
Jul 17 10:13:47.402938 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:47.402900    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed3c6f58-5662-46ec-9420-b8d927cb5290" containerName="revoke-keys"
Jul 17 10:13:47.406658 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:47.406627    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-v44mr"
Jul 17 10:13:47.413650 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:47.413611    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-v44mr"]
Jul 17 10:13:47.533143 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:47.533113    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fca12a67-1ffe-4d65-9ea8-ca446fca0ae9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-v44mr\" (UID: \"fca12a67-1ffe-4d65-9ea8-ca446fca0ae9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-v44mr"
Jul 17 10:13:47.633839 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:47.633803    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fca12a67-1ffe-4d65-9ea8-ca446fca0ae9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-v44mr\" (UID: \"fca12a67-1ffe-4d65-9ea8-ca446fca0ae9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-v44mr"
Jul 17 10:13:47.634490 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:47.634470    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fca12a67-1ffe-4d65-9ea8-ca446fca0ae9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-v44mr\" (UID: \"fca12a67-1ffe-4d65-9ea8-ca446fca0ae9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-v44mr"
Jul 17 10:13:47.721817 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:47.721772    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-v44mr"
Jul 17 10:13:47.827469 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:47.827438    2574 generic.go:356] "Generic (PLEG): container finished" podID="85b50483-a861-4c9c-a8e3-4b66857e2a04" containerID="f7d582511b485dfb2ecc65281bfc0e42a19703f50ae3765965168466a5ec75d4" exitCode=6
Jul 17 10:13:47.827667 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:47.827535    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tfdld" event={"ID":"85b50483-a861-4c9c-a8e3-4b66857e2a04","Type":"ContainerDied","Data":"f7d582511b485dfb2ecc65281bfc0e42a19703f50ae3765965168466a5ec75d4"}
Jul 17 10:13:47.827667 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:47.827598    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tfdld" event={"ID":"85b50483-a861-4c9c-a8e3-4b66857e2a04","Type":"ContainerStarted","Data":"78a5758d8d0a2e136d05f05a724c7b275a26e23f71ecf9e08baacca437117083"}
Jul 17 10:13:47.827806 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:47.827776    2574 scope.go:117] "RemoveContainer" containerID="f7d582511b485dfb2ecc65281bfc0e42a19703f50ae3765965168466a5ec75d4"
Jul 17 10:13:47.870571 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:47.870548    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-v44mr"]
Jul 17 10:13:47.872805 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:13:47.872778    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfca12a67_1ffe_4d65_9ea8_ca446fca0ae9.slice/crio-46aa7de09cc51d9035f98a565f7e7cb7df40d1a7bffd056b119b2e8ffb31969d WatchSource:0}: Error finding container 46aa7de09cc51d9035f98a565f7e7cb7df40d1a7bffd056b119b2e8ffb31969d: Status 404 returned error can't find the container with id 46aa7de09cc51d9035f98a565f7e7cb7df40d1a7bffd056b119b2e8ffb31969d
Jul 17 10:13:48.835136 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:48.835102    2574 generic.go:356] "Generic (PLEG): container finished" podID="fca12a67-1ffe-4d65-9ea8-ca446fca0ae9" containerID="4a92a5694d97bee61f5a7d6b43daeafc059bb131e9b3479859154bf5e111bfaf" exitCode=6
Jul 17 10:13:48.835582 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:48.835190    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-v44mr" event={"ID":"fca12a67-1ffe-4d65-9ea8-ca446fca0ae9","Type":"ContainerDied","Data":"4a92a5694d97bee61f5a7d6b43daeafc059bb131e9b3479859154bf5e111bfaf"}
Jul 17 10:13:48.835582 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:48.835281    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-v44mr" event={"ID":"fca12a67-1ffe-4d65-9ea8-ca446fca0ae9","Type":"ContainerStarted","Data":"46aa7de09cc51d9035f98a565f7e7cb7df40d1a7bffd056b119b2e8ffb31969d"}
Jul 17 10:13:48.835582 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:48.835376    2574 scope.go:117] "RemoveContainer" containerID="4a92a5694d97bee61f5a7d6b43daeafc059bb131e9b3479859154bf5e111bfaf"
Jul 17 10:13:48.837232 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:48.837211    2574 generic.go:356] "Generic (PLEG): container finished" podID="85b50483-a861-4c9c-a8e3-4b66857e2a04" containerID="f1dd5e3046625f7e177a1c81239bb51ce42fd5dd7b70668b8e3371f1ee7094d1" exitCode=6
Jul 17 10:13:48.837333 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:48.837271    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tfdld" event={"ID":"85b50483-a861-4c9c-a8e3-4b66857e2a04","Type":"ContainerDied","Data":"f1dd5e3046625f7e177a1c81239bb51ce42fd5dd7b70668b8e3371f1ee7094d1"}
Jul 17 10:13:48.837333 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:48.837319    2574 scope.go:117] "RemoveContainer" containerID="f7d582511b485dfb2ecc65281bfc0e42a19703f50ae3765965168466a5ec75d4"
Jul 17 10:13:48.837515 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:48.837496    2574 scope.go:117] "RemoveContainer" containerID="f1dd5e3046625f7e177a1c81239bb51ce42fd5dd7b70668b8e3371f1ee7094d1"
Jul 17 10:13:48.837709 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:13:48.837688    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-tfdld_odh-ai-gateway-infra(85b50483-a861-4c9c-a8e3-4b66857e2a04)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tfdld" podUID="85b50483-a861-4c9c-a8e3-4b66857e2a04"
Jul 17 10:13:49.844862 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:49.844763    2574 generic.go:356] "Generic (PLEG): container finished" podID="fca12a67-1ffe-4d65-9ea8-ca446fca0ae9" containerID="c04ed7af2ad711378cc5aa7386e548e85996323ae153b952362042857acd29d3" exitCode=6
Jul 17 10:13:49.844862 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:49.844843    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-v44mr" event={"ID":"fca12a67-1ffe-4d65-9ea8-ca446fca0ae9","Type":"ContainerDied","Data":"c04ed7af2ad711378cc5aa7386e548e85996323ae153b952362042857acd29d3"}
Jul 17 10:13:49.845394 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:49.844907    2574 scope.go:117] "RemoveContainer" containerID="4a92a5694d97bee61f5a7d6b43daeafc059bb131e9b3479859154bf5e111bfaf"
Jul 17 10:13:49.845394 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:49.845192    2574 scope.go:117] "RemoveContainer" containerID="c04ed7af2ad711378cc5aa7386e548e85996323ae153b952362042857acd29d3"
Jul 17 10:13:49.845523 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:13:49.845498    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-v44mr_odh-ai-gateway-infra(fca12a67-1ffe-4d65-9ea8-ca446fca0ae9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-v44mr" podUID="fca12a67-1ffe-4d65-9ea8-ca446fca0ae9"
Jul 17 10:13:49.846898 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:49.846883    2574 scope.go:117] "RemoveContainer" containerID="f1dd5e3046625f7e177a1c81239bb51ce42fd5dd7b70668b8e3371f1ee7094d1"
Jul 17 10:13:49.847062 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:13:49.847045    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-tfdld_odh-ai-gateway-infra(85b50483-a861-4c9c-a8e3-4b66857e2a04)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tfdld" podUID="85b50483-a861-4c9c-a8e3-4b66857e2a04"
Jul 17 10:13:50.854599 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:50.854573    2574 scope.go:117] "RemoveContainer" containerID="c04ed7af2ad711378cc5aa7386e548e85996323ae153b952362042857acd29d3"
Jul 17 10:13:50.854964 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:13:50.854786    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-v44mr_odh-ai-gateway-infra(fca12a67-1ffe-4d65-9ea8-ca446fca0ae9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-v44mr" podUID="fca12a67-1ffe-4d65-9ea8-ca446fca0ae9"
Jul 17 10:13:50.892203 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:50.892169    2574 scope.go:117] "RemoveContainer" containerID="4d9e26b6552cabcccccb1e422d67588129a75921175595b73eae44d4b09bf662"
Jul 17 10:13:51.862027 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:51.861935    2574 generic.go:356] "Generic (PLEG): container finished" podID="299b9375-bc51-452a-b3ad-fd1f659454ef" containerID="03b713f19ab92766181d1bfb2aa8061990ccd8102d0322ad40f958db9dd87fce" exitCode=6
Jul 17 10:13:51.862027 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:51.862011    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-ljz26" event={"ID":"299b9375-bc51-452a-b3ad-fd1f659454ef","Type":"ContainerDied","Data":"03b713f19ab92766181d1bfb2aa8061990ccd8102d0322ad40f958db9dd87fce"}
Jul 17 10:13:51.862499 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:51.862100    2574 scope.go:117] "RemoveContainer" containerID="4d9e26b6552cabcccccb1e422d67588129a75921175595b73eae44d4b09bf662"
Jul 17 10:13:51.862499 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:51.862482    2574 scope.go:117] "RemoveContainer" containerID="03b713f19ab92766181d1bfb2aa8061990ccd8102d0322ad40f958db9dd87fce"
Jul 17 10:13:51.862755 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:13:51.862736    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-ljz26_odh-ai-gateway-infra(299b9375-bc51-452a-b3ad-fd1f659454ef)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-ljz26" podUID="299b9375-bc51-452a-b3ad-fd1f659454ef"
Jul 17 10:13:52.888479 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:52.888444    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-ljz26"]
Jul 17 10:13:53.042693 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:53.042663    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-ljz26"
Jul 17 10:13:53.189516 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:53.189465    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/299b9375-bc51-452a-b3ad-fd1f659454ef-maas-api-service-ca\") pod \"299b9375-bc51-452a-b3ad-fd1f659454ef\" (UID: \"299b9375-bc51-452a-b3ad-fd1f659454ef\") "
Jul 17 10:13:53.189825 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:53.189804    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/299b9375-bc51-452a-b3ad-fd1f659454ef-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "299b9375-bc51-452a-b3ad-fd1f659454ef" (UID: "299b9375-bc51-452a-b3ad-fd1f659454ef"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:13:53.291052 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:53.291017    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/299b9375-bc51-452a-b3ad-fd1f659454ef-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:13:53.875275 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:53.875230    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-ljz26"
Jul 17 10:13:53.875275 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:53.875265    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-ljz26" event={"ID":"299b9375-bc51-452a-b3ad-fd1f659454ef","Type":"ContainerDied","Data":"ace0cafda25fba4957d63aa7e8edf9bea151269e07d3c2589963b54a41dd9f14"}
Jul 17 10:13:53.875520 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:53.875327    2574 scope.go:117] "RemoveContainer" containerID="03b713f19ab92766181d1bfb2aa8061990ccd8102d0322ad40f958db9dd87fce"
Jul 17 10:13:53.900745 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:53.900710    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-ljz26"]
Jul 17 10:13:53.904814 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:53.904791    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-ljz26"]
Jul 17 10:13:54.203777 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:54.203734    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21e67a-fqs97"]
Jul 17 10:13:54.204461 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:54.204437    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="299b9375-bc51-452a-b3ad-fd1f659454ef" containerName="revoke-keys"
Jul 17 10:13:54.204560 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:54.204463    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="299b9375-bc51-452a-b3ad-fd1f659454ef" containerName="revoke-keys"
Jul 17 10:13:54.204560 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:54.204487    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="299b9375-bc51-452a-b3ad-fd1f659454ef" containerName="revoke-keys"
Jul 17 10:13:54.204560 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:54.204496    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="299b9375-bc51-452a-b3ad-fd1f659454ef" containerName="revoke-keys"
Jul 17 10:13:54.204560 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:54.204510    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="299b9375-bc51-452a-b3ad-fd1f659454ef" containerName="revoke-keys"
Jul 17 10:13:54.204560 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:54.204518    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="299b9375-bc51-452a-b3ad-fd1f659454ef" containerName="revoke-keys"
Jul 17 10:13:54.204718 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:54.204633    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="299b9375-bc51-452a-b3ad-fd1f659454ef" containerName="revoke-keys"
Jul 17 10:13:54.204718 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:54.204652    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="299b9375-bc51-452a-b3ad-fd1f659454ef" containerName="revoke-keys"
Jul 17 10:13:54.209915 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:54.209892    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21e67a-fqs97"
Jul 17 10:13:54.215447 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:54.215417    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21e67a-fqs97"]
Jul 17 10:13:54.402617 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:54.402575    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c29a17d5-9aea-4e31-95ed-29f0b338f3f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-21e67a-fqs97\" (UID: \"c29a17d5-9aea-4e31-95ed-29f0b338f3f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21e67a-fqs97"
Jul 17 10:13:54.504186 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:54.504104    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c29a17d5-9aea-4e31-95ed-29f0b338f3f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-21e67a-fqs97\" (UID: \"c29a17d5-9aea-4e31-95ed-29f0b338f3f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21e67a-fqs97"
Jul 17 10:13:54.504955 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:54.504926    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c29a17d5-9aea-4e31-95ed-29f0b338f3f4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-21e67a-fqs97\" (UID: \"c29a17d5-9aea-4e31-95ed-29f0b338f3f4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21e67a-fqs97"
Jul 17 10:13:54.527206 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:54.527163    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21e67a-fqs97"
Jul 17 10:13:54.680431 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:54.680397    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21e67a-fqs97"]
Jul 17 10:13:54.681100 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:13:54.681069    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc29a17d5_9aea_4e31_95ed_29f0b338f3f4.slice/crio-bf3403c046fff54de1a7c1c704867d1b3a05f54f7503ea434422c8bed4b26a9d WatchSource:0}: Error finding container bf3403c046fff54de1a7c1c704867d1b3a05f54f7503ea434422c8bed4b26a9d: Status 404 returned error can't find the container with id bf3403c046fff54de1a7c1c704867d1b3a05f54f7503ea434422c8bed4b26a9d
Jul 17 10:13:54.883100 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:54.883069    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21e67a-fqs97" event={"ID":"c29a17d5-9aea-4e31-95ed-29f0b338f3f4","Type":"ContainerStarted","Data":"bf3403c046fff54de1a7c1c704867d1b3a05f54f7503ea434422c8bed4b26a9d"}
Jul 17 10:13:54.898545 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:54.898506    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="299b9375-bc51-452a-b3ad-fd1f659454ef" path="/var/lib/kubelet/pods/299b9375-bc51-452a-b3ad-fd1f659454ef/volumes"
Jul 17 10:13:55.899698 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:55.899609    2574 generic.go:356] "Generic (PLEG): container finished" podID="c29a17d5-9aea-4e31-95ed-29f0b338f3f4" containerID="cecfd3fe5f0d655b6be7c5c9522ca800e3752584e82ee2b2285e033cee29b216" exitCode=0
Jul 17 10:13:55.900118 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:55.899693    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21e67a-fqs97" event={"ID":"c29a17d5-9aea-4e31-95ed-29f0b338f3f4","Type":"ContainerDied","Data":"cecfd3fe5f0d655b6be7c5c9522ca800e3752584e82ee2b2285e033cee29b216"}
Jul 17 10:13:57.046932 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:57.046902    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21e67a-fqs97"
Jul 17 10:13:57.131156 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:57.131104    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c29a17d5-9aea-4e31-95ed-29f0b338f3f4-maas-api-service-ca\") pod \"c29a17d5-9aea-4e31-95ed-29f0b338f3f4\" (UID: \"c29a17d5-9aea-4e31-95ed-29f0b338f3f4\") "
Jul 17 10:13:57.131590 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:57.131560    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c29a17d5-9aea-4e31-95ed-29f0b338f3f4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c29a17d5-9aea-4e31-95ed-29f0b338f3f4" (UID: "c29a17d5-9aea-4e31-95ed-29f0b338f3f4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:13:57.232582 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:57.232542    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c29a17d5-9aea-4e31-95ed-29f0b338f3f4-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:13:57.912005 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:57.911966    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21e67a-fqs97"
Jul 17 10:13:57.912189 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:57.912006    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-21e67a-fqs97" event={"ID":"c29a17d5-9aea-4e31-95ed-29f0b338f3f4","Type":"ContainerDied","Data":"bf3403c046fff54de1a7c1c704867d1b3a05f54f7503ea434422c8bed4b26a9d"}
Jul 17 10:13:57.912189 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:13:57.912052    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bf3403c046fff54de1a7c1c704867d1b3a05f54f7503ea434422c8bed4b26a9d"
Jul 17 10:14:02.894941 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:02.894915    2574 scope.go:117] "RemoveContainer" containerID="c04ed7af2ad711378cc5aa7386e548e85996323ae153b952362042857acd29d3"
Jul 17 10:14:03.892042 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:03.891958    2574 scope.go:117] "RemoveContainer" containerID="f1dd5e3046625f7e177a1c81239bb51ce42fd5dd7b70668b8e3371f1ee7094d1"
Jul 17 10:14:03.949029 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:03.949000    2574 generic.go:356] "Generic (PLEG): container finished" podID="fca12a67-1ffe-4d65-9ea8-ca446fca0ae9" containerID="946cf51c1b1e6debd984042fc1d5c54bf4647d5d238d2ae5125a304afdcf6a40" exitCode=6
Jul 17 10:14:03.949389 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:03.949081    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-v44mr" event={"ID":"fca12a67-1ffe-4d65-9ea8-ca446fca0ae9","Type":"ContainerDied","Data":"946cf51c1b1e6debd984042fc1d5c54bf4647d5d238d2ae5125a304afdcf6a40"}
Jul 17 10:14:03.949389 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:03.949150    2574 scope.go:117] "RemoveContainer" containerID="c04ed7af2ad711378cc5aa7386e548e85996323ae153b952362042857acd29d3"
Jul 17 10:14:03.949512 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:03.949500    2574 scope.go:117] "RemoveContainer" containerID="946cf51c1b1e6debd984042fc1d5c54bf4647d5d238d2ae5125a304afdcf6a40"
Jul 17 10:14:03.949790 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:14:03.949758    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-v44mr_odh-ai-gateway-infra(fca12a67-1ffe-4d65-9ea8-ca446fca0ae9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-v44mr" podUID="fca12a67-1ffe-4d65-9ea8-ca446fca0ae9"
Jul 17 10:14:04.958182 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:04.958151    2574 generic.go:356] "Generic (PLEG): container finished" podID="85b50483-a861-4c9c-a8e3-4b66857e2a04" containerID="05767a9c41780768d02820552955ad8fc8f8b8c585b18282501a6e16ddf694eb" exitCode=6
Jul 17 10:14:04.958629 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:04.958226    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tfdld" event={"ID":"85b50483-a861-4c9c-a8e3-4b66857e2a04","Type":"ContainerDied","Data":"05767a9c41780768d02820552955ad8fc8f8b8c585b18282501a6e16ddf694eb"}
Jul 17 10:14:04.958629 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:04.958310    2574 scope.go:117] "RemoveContainer" containerID="f1dd5e3046625f7e177a1c81239bb51ce42fd5dd7b70668b8e3371f1ee7094d1"
Jul 17 10:14:04.958629 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:04.958572    2574 scope.go:117] "RemoveContainer" containerID="05767a9c41780768d02820552955ad8fc8f8b8c585b18282501a6e16ddf694eb"
Jul 17 10:14:04.958847 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:14:04.958823    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-tfdld_odh-ai-gateway-infra(85b50483-a861-4c9c-a8e3-4b66857e2a04)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tfdld" podUID="85b50483-a861-4c9c-a8e3-4b66857e2a04"
Jul 17 10:14:04.980715 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:04.980690    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-v44mr"]
Jul 17 10:14:05.140683 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:05.140659    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-v44mr"
Jul 17 10:14:05.211559 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:05.211466    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fca12a67-1ffe-4d65-9ea8-ca446fca0ae9-maas-api-service-ca\") pod \"fca12a67-1ffe-4d65-9ea8-ca446fca0ae9\" (UID: \"fca12a67-1ffe-4d65-9ea8-ca446fca0ae9\") "
Jul 17 10:14:05.211901 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:05.211875    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fca12a67-1ffe-4d65-9ea8-ca446fca0ae9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fca12a67-1ffe-4d65-9ea8-ca446fca0ae9" (UID: "fca12a67-1ffe-4d65-9ea8-ca446fca0ae9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:14:05.313296 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:05.313265    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fca12a67-1ffe-4d65-9ea8-ca446fca0ae9-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:14:05.965672 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:05.965641    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-v44mr"
Jul 17 10:14:05.966157 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:05.965639    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-v44mr" event={"ID":"fca12a67-1ffe-4d65-9ea8-ca446fca0ae9","Type":"ContainerDied","Data":"46aa7de09cc51d9035f98a565f7e7cb7df40d1a7bffd056b119b2e8ffb31969d"}
Jul 17 10:14:05.966157 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:05.965792    2574 scope.go:117] "RemoveContainer" containerID="946cf51c1b1e6debd984042fc1d5c54bf4647d5d238d2ae5125a304afdcf6a40"
Jul 17 10:14:05.984209 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:05.984181    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tfdld"]
Jul 17 10:14:05.989226 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:05.989203    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-v44mr"]
Jul 17 10:14:05.993950 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:05.993929    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-v44mr"]
Jul 17 10:14:06.139290 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:06.139264    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tfdld"
Jul 17 10:14:06.222883 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:06.222801    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85b50483-a861-4c9c-a8e3-4b66857e2a04-maas-api-service-ca\") pod \"85b50483-a861-4c9c-a8e3-4b66857e2a04\" (UID: \"85b50483-a861-4c9c-a8e3-4b66857e2a04\") "
Jul 17 10:14:06.223205 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:06.223177    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85b50483-a861-4c9c-a8e3-4b66857e2a04-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "85b50483-a861-4c9c-a8e3-4b66857e2a04" (UID: "85b50483-a861-4c9c-a8e3-4b66857e2a04"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:14:06.323726 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:06.323690    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/85b50483-a861-4c9c-a8e3-4b66857e2a04-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:14:06.897912 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:06.897875    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fca12a67-1ffe-4d65-9ea8-ca446fca0ae9" path="/var/lib/kubelet/pods/fca12a67-1ffe-4d65-9ea8-ca446fca0ae9/volumes"
Jul 17 10:14:06.975429 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:06.975400    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tfdld"
Jul 17 10:14:06.975848 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:06.975445    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tfdld" event={"ID":"85b50483-a861-4c9c-a8e3-4b66857e2a04","Type":"ContainerDied","Data":"78a5758d8d0a2e136d05f05a724c7b275a26e23f71ecf9e08baacca437117083"}
Jul 17 10:14:06.975848 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:06.975516    2574 scope.go:117] "RemoveContainer" containerID="05767a9c41780768d02820552955ad8fc8f8b8c585b18282501a6e16ddf694eb"
Jul 17 10:14:06.995272 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:06.995233    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tfdld"]
Jul 17 10:14:07.001431 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:07.001410    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-tfdld"]
Jul 17 10:14:08.898834 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:08.898795    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85b50483-a861-4c9c-a8e3-4b66857e2a04" path="/var/lib/kubelet/pods/85b50483-a861-4c9c-a8e3-4b66857e2a04/volumes"
Jul 17 10:14:20.883097 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:20.883058    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c996b-xms8l"]
Jul 17 10:14:20.883541 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:20.883527    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85b50483-a861-4c9c-a8e3-4b66857e2a04" containerName="revoke-keys"
Jul 17 10:14:20.883541 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:20.883542    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="85b50483-a861-4c9c-a8e3-4b66857e2a04" containerName="revoke-keys"
Jul 17 10:14:20.883622 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:20.883554    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85b50483-a861-4c9c-a8e3-4b66857e2a04" containerName="revoke-keys"
Jul 17 10:14:20.883622 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:20.883560    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="85b50483-a861-4c9c-a8e3-4b66857e2a04" containerName="revoke-keys"
Jul 17 10:14:20.883622 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:20.883572    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fca12a67-1ffe-4d65-9ea8-ca446fca0ae9" containerName="revoke-keys"
Jul 17 10:14:20.883622 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:20.883577    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="fca12a67-1ffe-4d65-9ea8-ca446fca0ae9" containerName="revoke-keys"
Jul 17 10:14:20.883622 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:20.883600    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c29a17d5-9aea-4e31-95ed-29f0b338f3f4" containerName="revoke-keys"
Jul 17 10:14:20.883622 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:20.883605    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c29a17d5-9aea-4e31-95ed-29f0b338f3f4" containerName="revoke-keys"
Jul 17 10:14:20.883622 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:20.883611    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fca12a67-1ffe-4d65-9ea8-ca446fca0ae9" containerName="revoke-keys"
Jul 17 10:14:20.883622 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:20.883616    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="fca12a67-1ffe-4d65-9ea8-ca446fca0ae9" containerName="revoke-keys"
Jul 17 10:14:20.883864 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:20.883674    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="85b50483-a861-4c9c-a8e3-4b66857e2a04" containerName="revoke-keys"
Jul 17 10:14:20.883864 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:20.883680    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c29a17d5-9aea-4e31-95ed-29f0b338f3f4" containerName="revoke-keys"
Jul 17 10:14:20.883864 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:20.883688    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="fca12a67-1ffe-4d65-9ea8-ca446fca0ae9" containerName="revoke-keys"
Jul 17 10:14:20.883864 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:20.883693    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="85b50483-a861-4c9c-a8e3-4b66857e2a04" containerName="revoke-keys"
Jul 17 10:14:20.883864 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:20.883699    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="fca12a67-1ffe-4d65-9ea8-ca446fca0ae9" containerName="revoke-keys"
Jul 17 10:14:20.883864 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:20.883706    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="299b9375-bc51-452a-b3ad-fd1f659454ef" containerName="revoke-keys"
Jul 17 10:14:20.888206 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:20.888188    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c996b-xms8l"
Jul 17 10:14:20.891631 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:20.891599    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:14:20.901923 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:20.901894    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c996b-xms8l"]
Jul 17 10:14:20.965298 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:20.965224    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63a05154-b1a1-4aa8-a750-fe842b3b139d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-1c996b-xms8l\" (UID: \"63a05154-b1a1-4aa8-a750-fe842b3b139d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c996b-xms8l"
Jul 17 10:14:21.066818 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:21.066778    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63a05154-b1a1-4aa8-a750-fe842b3b139d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-1c996b-xms8l\" (UID: \"63a05154-b1a1-4aa8-a750-fe842b3b139d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c996b-xms8l"
Jul 17 10:14:21.067456 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:21.067434    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63a05154-b1a1-4aa8-a750-fe842b3b139d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-1c996b-xms8l\" (UID: \"63a05154-b1a1-4aa8-a750-fe842b3b139d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c996b-xms8l"
Jul 17 10:14:21.202578 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:21.202538    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c996b-xms8l"
Jul 17 10:14:21.352591 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:21.352560    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c996b-xms8l"]
Jul 17 10:14:21.354802 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:14:21.354775    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63a05154_b1a1_4aa8_a750_fe842b3b139d.slice/crio-b84df85e80162d9b486564554003d188bc0a31a12f0ee7ef258e110aaec5a68c WatchSource:0}: Error finding container b84df85e80162d9b486564554003d188bc0a31a12f0ee7ef258e110aaec5a68c: Status 404 returned error can't find the container with id b84df85e80162d9b486564554003d188bc0a31a12f0ee7ef258e110aaec5a68c
Jul 17 10:14:22.066618 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:22.066580    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c996b-xms8l" event={"ID":"63a05154-b1a1-4aa8-a750-fe842b3b139d","Type":"ContainerStarted","Data":"86a372c80b063145934b0ca4480d185a13d0d174e41a815cce53a88d0bab5741"}
Jul 17 10:14:22.067000 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:22.066643    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c996b-xms8l" event={"ID":"63a05154-b1a1-4aa8-a750-fe842b3b139d","Type":"ContainerStarted","Data":"b84df85e80162d9b486564554003d188bc0a31a12f0ee7ef258e110aaec5a68c"}
Jul 17 10:14:23.077506 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:23.077473    2574 generic.go:356] "Generic (PLEG): container finished" podID="63a05154-b1a1-4aa8-a750-fe842b3b139d" containerID="86a372c80b063145934b0ca4480d185a13d0d174e41a815cce53a88d0bab5741" exitCode=0
Jul 17 10:14:23.077917 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:23.077517    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c996b-xms8l" event={"ID":"63a05154-b1a1-4aa8-a750-fe842b3b139d","Type":"ContainerDied","Data":"86a372c80b063145934b0ca4480d185a13d0d174e41a815cce53a88d0bab5741"}
Jul 17 10:14:24.219445 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:24.219419    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c996b-xms8l"
Jul 17 10:14:24.300954 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:24.300927    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63a05154-b1a1-4aa8-a750-fe842b3b139d-maas-api-service-ca\") pod \"63a05154-b1a1-4aa8-a750-fe842b3b139d\" (UID: \"63a05154-b1a1-4aa8-a750-fe842b3b139d\") "
Jul 17 10:14:24.301387 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:24.301354    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63a05154-b1a1-4aa8-a750-fe842b3b139d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "63a05154-b1a1-4aa8-a750-fe842b3b139d" (UID: "63a05154-b1a1-4aa8-a750-fe842b3b139d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:14:24.402112 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:24.402018    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/63a05154-b1a1-4aa8-a750-fe842b3b139d-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:14:25.090203 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:25.090160    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c996b-xms8l" event={"ID":"63a05154-b1a1-4aa8-a750-fe842b3b139d","Type":"ContainerDied","Data":"b84df85e80162d9b486564554003d188bc0a31a12f0ee7ef258e110aaec5a68c"}
Jul 17 10:14:25.090203 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:25.090189    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-1c996b-xms8l"
Jul 17 10:14:25.090203 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:25.090212    2574 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b84df85e80162d9b486564554003d188bc0a31a12f0ee7ef258e110aaec5a68c"
Jul 17 10:14:46.658765 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:46.658725    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"]
Jul 17 10:14:46.659303 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:46.658962    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl" podUID="8b0de510-d71b-4b55-8217-3c9b8428668b" containerName="istio-proxy" containerID="cri-o://34436f107a1e0fd19329b72f02df532bc2553cb0244beeb26d5607a14b2da43f" gracePeriod=30
Jul 17 10:14:51.931444 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:51.931374    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:14:51.979286 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:51.979226    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-credential-socket\") pod \"8b0de510-d71b-4b55-8217-3c9b8428668b\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") "
Jul 17 10:14:51.979466 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:51.979300    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b0de510-d71b-4b55-8217-3c9b8428668b-istio-podinfo\") pod \"8b0de510-d71b-4b55-8217-3c9b8428668b\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") "
Jul 17 10:14:51.979466 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:51.979342    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-istio-data\") pod \"8b0de510-d71b-4b55-8217-3c9b8428668b\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") "
Jul 17 10:14:51.979466 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:51.979392    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b0de510-d71b-4b55-8217-3c9b8428668b-istio-token\") pod \"8b0de510-d71b-4b55-8217-3c9b8428668b\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") "
Jul 17 10:14:51.979628 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:51.979468    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b0de510-d71b-4b55-8217-3c9b8428668b-istiod-ca-cert\") pod \"8b0de510-d71b-4b55-8217-3c9b8428668b\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") "
Jul 17 10:14:51.979628 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:51.979490    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8b0de510-d71b-4b55-8217-3c9b8428668b" (UID: "8b0de510-d71b-4b55-8217-3c9b8428668b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:14:51.979628 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:51.979534    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-workload-socket\") pod \"8b0de510-d71b-4b55-8217-3c9b8428668b\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") "
Jul 17 10:14:51.979628 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:51.979589    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-istio-envoy\") pod \"8b0de510-d71b-4b55-8217-3c9b8428668b\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") "
Jul 17 10:14:51.979628 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:51.979613    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-workload-certs\") pod \"8b0de510-d71b-4b55-8217-3c9b8428668b\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") "
Jul 17 10:14:51.979897 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:51.979644    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfpq8\" (UniqueName: \"kubernetes.io/projected/8b0de510-d71b-4b55-8217-3c9b8428668b-kube-api-access-bfpq8\") pod \"8b0de510-d71b-4b55-8217-3c9b8428668b\" (UID: \"8b0de510-d71b-4b55-8217-3c9b8428668b\") "
Jul 17 10:14:51.979897 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:51.979739    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8b0de510-d71b-4b55-8217-3c9b8428668b" (UID: "8b0de510-d71b-4b55-8217-3c9b8428668b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:14:51.979897 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:51.979878    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b0de510-d71b-4b55-8217-3c9b8428668b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8b0de510-d71b-4b55-8217-3c9b8428668b" (UID: "8b0de510-d71b-4b55-8217-3c9b8428668b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:14:51.980775 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:51.980122    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-credential-socket\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:14:51.980775 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:51.980147    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8b0de510-d71b-4b55-8217-3c9b8428668b-istiod-ca-cert\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:14:51.980775 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:51.980161    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-workload-socket\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:14:51.980775 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:51.980399    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8b0de510-d71b-4b55-8217-3c9b8428668b" (UID: "8b0de510-d71b-4b55-8217-3c9b8428668b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:14:51.980775 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:51.980459    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-istio-data" (OuterVolumeSpecName: "istio-data") pod "8b0de510-d71b-4b55-8217-3c9b8428668b" (UID: "8b0de510-d71b-4b55-8217-3c9b8428668b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:14:51.982429 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:51.982389    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b0de510-d71b-4b55-8217-3c9b8428668b-istio-token" (OuterVolumeSpecName: "istio-token") pod "8b0de510-d71b-4b55-8217-3c9b8428668b" (UID: "8b0de510-d71b-4b55-8217-3c9b8428668b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:14:51.982546 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:51.982512    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b0de510-d71b-4b55-8217-3c9b8428668b-kube-api-access-bfpq8" (OuterVolumeSpecName: "kube-api-access-bfpq8") pod "8b0de510-d71b-4b55-8217-3c9b8428668b" (UID: "8b0de510-d71b-4b55-8217-3c9b8428668b"). InnerVolumeSpecName "kube-api-access-bfpq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:14:51.982639 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:51.982614    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8b0de510-d71b-4b55-8217-3c9b8428668b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8b0de510-d71b-4b55-8217-3c9b8428668b" (UID: "8b0de510-d71b-4b55-8217-3c9b8428668b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 10:14:51.982968 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:51.982936    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8b0de510-d71b-4b55-8217-3c9b8428668b" (UID: "8b0de510-d71b-4b55-8217-3c9b8428668b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:14:52.081127 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:52.081093    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-istio-envoy\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:14:52.081127 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:52.081129    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-workload-certs\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:14:52.081360 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:52.081139    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfpq8\" (UniqueName: \"kubernetes.io/projected/8b0de510-d71b-4b55-8217-3c9b8428668b-kube-api-access-bfpq8\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:14:52.081360 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:52.081150    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8b0de510-d71b-4b55-8217-3c9b8428668b-istio-podinfo\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:14:52.081360 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:52.081163    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8b0de510-d71b-4b55-8217-3c9b8428668b-istio-data\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:14:52.081360 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:52.081175    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8b0de510-d71b-4b55-8217-3c9b8428668b-istio-token\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:14:52.245741 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:52.245649    2574 generic.go:356] "Generic (PLEG): container finished" podID="8b0de510-d71b-4b55-8217-3c9b8428668b" containerID="34436f107a1e0fd19329b72f02df532bc2553cb0244beeb26d5607a14b2da43f" exitCode=0
Jul 17 10:14:52.245741 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:52.245715    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl" event={"ID":"8b0de510-d71b-4b55-8217-3c9b8428668b","Type":"ContainerDied","Data":"34436f107a1e0fd19329b72f02df532bc2553cb0244beeb26d5607a14b2da43f"}
Jul 17 10:14:52.245942 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:52.245782    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl" event={"ID":"8b0de510-d71b-4b55-8217-3c9b8428668b","Type":"ContainerDied","Data":"eff4bfda427667784ad30b70b39da3e9d6186407e39f339bab00a01a0020db6d"}
Jul 17 10:14:52.245942 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:52.245786    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"
Jul 17 10:14:52.245942 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:52.245809    2574 scope.go:117] "RemoveContainer" containerID="34436f107a1e0fd19329b72f02df532bc2553cb0244beeb26d5607a14b2da43f"
Jul 17 10:14:52.256215 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:52.256194    2574 scope.go:117] "RemoveContainer" containerID="aaa0321094ca1b90a363d9e2f6a4266e151d629cf411dec23c9df9b95bb8f7e0"
Jul 17 10:14:52.264733 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:52.264714    2574 scope.go:117] "RemoveContainer" containerID="34436f107a1e0fd19329b72f02df532bc2553cb0244beeb26d5607a14b2da43f"
Jul 17 10:14:52.264987 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:14:52.264971    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34436f107a1e0fd19329b72f02df532bc2553cb0244beeb26d5607a14b2da43f\": container with ID starting with 34436f107a1e0fd19329b72f02df532bc2553cb0244beeb26d5607a14b2da43f not found: ID does not exist" containerID="34436f107a1e0fd19329b72f02df532bc2553cb0244beeb26d5607a14b2da43f"
Jul 17 10:14:52.265031 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:52.264996    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34436f107a1e0fd19329b72f02df532bc2553cb0244beeb26d5607a14b2da43f"} err="failed to get container status \"34436f107a1e0fd19329b72f02df532bc2553cb0244beeb26d5607a14b2da43f\": rpc error: code = NotFound desc = could not find container \"34436f107a1e0fd19329b72f02df532bc2553cb0244beeb26d5607a14b2da43f\": container with ID starting with 34436f107a1e0fd19329b72f02df532bc2553cb0244beeb26d5607a14b2da43f not found: ID does not exist"
Jul 17 10:14:52.265031 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:52.265013    2574 scope.go:117] "RemoveContainer" containerID="aaa0321094ca1b90a363d9e2f6a4266e151d629cf411dec23c9df9b95bb8f7e0"
Jul 17 10:14:52.265321 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:14:52.265299    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aaa0321094ca1b90a363d9e2f6a4266e151d629cf411dec23c9df9b95bb8f7e0\": container with ID starting with aaa0321094ca1b90a363d9e2f6a4266e151d629cf411dec23c9df9b95bb8f7e0 not found: ID does not exist" containerID="aaa0321094ca1b90a363d9e2f6a4266e151d629cf411dec23c9df9b95bb8f7e0"
Jul 17 10:14:52.265358 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:52.265330    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aaa0321094ca1b90a363d9e2f6a4266e151d629cf411dec23c9df9b95bb8f7e0"} err="failed to get container status \"aaa0321094ca1b90a363d9e2f6a4266e151d629cf411dec23c9df9b95bb8f7e0\": rpc error: code = NotFound desc = could not find container \"aaa0321094ca1b90a363d9e2f6a4266e151d629cf411dec23c9df9b95bb8f7e0\": container with ID starting with aaa0321094ca1b90a363d9e2f6a4266e151d629cf411dec23c9df9b95bb8f7e0 not found: ID does not exist"
Jul 17 10:14:52.269294 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:52.269252    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"]
Jul 17 10:14:52.279888 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:52.279860    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-1c996b-openshift-default-9c559969f-22wnl"]
Jul 17 10:14:52.898587 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:52.898543    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b0de510-d71b-4b55-8217-3c9b8428668b" path="/var/lib/kubelet/pods/8b0de510-d71b-4b55-8217-3c9b8428668b/volumes"
Jul 17 10:14:56.233903 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.233864    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"]
Jul 17 10:14:56.234396 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.234380    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fca12a67-1ffe-4d65-9ea8-ca446fca0ae9" containerName="revoke-keys"
Jul 17 10:14:56.234449 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.234398    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="fca12a67-1ffe-4d65-9ea8-ca446fca0ae9" containerName="revoke-keys"
Jul 17 10:14:56.234449 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.234409    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b0de510-d71b-4b55-8217-3c9b8428668b" containerName="istio-proxy"
Jul 17 10:14:56.234449 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.234415    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b0de510-d71b-4b55-8217-3c9b8428668b" containerName="istio-proxy"
Jul 17 10:14:56.234449 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.234425    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85b50483-a861-4c9c-a8e3-4b66857e2a04" containerName="revoke-keys"
Jul 17 10:14:56.234449 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.234431    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="85b50483-a861-4c9c-a8e3-4b66857e2a04" containerName="revoke-keys"
Jul 17 10:14:56.234648 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.234453    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63a05154-b1a1-4aa8-a750-fe842b3b139d" containerName="revoke-keys"
Jul 17 10:14:56.234648 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.234459    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="63a05154-b1a1-4aa8-a750-fe842b3b139d" containerName="revoke-keys"
Jul 17 10:14:56.234648 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.234515    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="fca12a67-1ffe-4d65-9ea8-ca446fca0ae9" containerName="revoke-keys"
Jul 17 10:14:56.234648 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.234522    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="63a05154-b1a1-4aa8-a750-fe842b3b139d" containerName="revoke-keys"
Jul 17 10:14:56.234648 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.234531    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b0de510-d71b-4b55-8217-3c9b8428668b" containerName="istio-proxy"
Jul 17 10:14:56.234648 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.234538    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="85b50483-a861-4c9c-a8e3-4b66857e2a04" containerName="revoke-keys"
Jul 17 10:14:56.234648 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.234545    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b0de510-d71b-4b55-8217-3c9b8428668b" containerName="istio-proxy"
Jul 17 10:14:56.239022 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.239002    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.241890 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.241870    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-1baf67-openshift-default-dockercfg-qt8nd\""
Jul 17 10:14:56.250110 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.250087    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"]
Jul 17 10:14:56.323581 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.323544    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/723a906f-455e-470a-bce6-14ee71991758-istio-podinfo\") pod \"e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") " pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.323581 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.323586    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-istio-data\") pod \"e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") " pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.323789 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.323608    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-workload-socket\") pod \"e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") " pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.323789 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.323654    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdrv8\" (UniqueName: \"kubernetes.io/projected/723a906f-455e-470a-bce6-14ee71991758-kube-api-access-bdrv8\") pod \"e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") " pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.323789 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.323734    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-workload-certs\") pod \"e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") " pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.323789 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.323750    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/723a906f-455e-470a-bce6-14ee71991758-istiod-ca-cert\") pod \"e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") " pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.323789 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.323768    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-istio-envoy\") pod \"e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") " pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.323987 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.323801    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/723a906f-455e-470a-bce6-14ee71991758-istio-token\") pod \"e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") " pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.323987 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.323839    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-credential-socket\") pod \"e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") " pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.425352 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.425309    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-workload-certs\") pod \"e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") " pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.425352 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.425356    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/723a906f-455e-470a-bce6-14ee71991758-istiod-ca-cert\") pod \"e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") " pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.425609 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.425381    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-istio-envoy\") pod \"e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") " pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.425609 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.425413    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/723a906f-455e-470a-bce6-14ee71991758-istio-token\") pod \"e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") " pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.425609 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.425446    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-credential-socket\") pod \"e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") " pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.425609 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.425530    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/723a906f-455e-470a-bce6-14ee71991758-istio-podinfo\") pod \"e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") " pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.425609 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.425559    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-istio-data\") pod \"e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") " pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.425879 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.425595    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-workload-socket\") pod \"e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") " pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.425879 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.425695    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdrv8\" (UniqueName: \"kubernetes.io/projected/723a906f-455e-470a-bce6-14ee71991758-kube-api-access-bdrv8\") pod \"e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") " pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.425879 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.425808    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-workload-certs\") pod \"e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") " pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.425879 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.425847    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-credential-socket\") pod \"e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") " pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.426116 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.426094    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-workload-socket\") pod \"e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") " pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.426170 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.426120    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-istio-data\") pod \"e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") " pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.426218 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.426191    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/723a906f-455e-470a-bce6-14ee71991758-istiod-ca-cert\") pod \"e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") " pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.427917 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.427895    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-istio-envoy\") pod \"e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") " pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.428189 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.428169    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/723a906f-455e-470a-bce6-14ee71991758-istio-podinfo\") pod \"e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") " pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.434087 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.434062    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/723a906f-455e-470a-bce6-14ee71991758-istio-token\") pod \"e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") " pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.434223 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.434206    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdrv8\" (UniqueName: \"kubernetes.io/projected/723a906f-455e-470a-bce6-14ee71991758-kube-api-access-bdrv8\") pod \"e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") " pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.550330 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.550226    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:56.702221 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.702185    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"]
Jul 17 10:14:56.703467 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:14:56.703424    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod723a906f_455e_470a_bce6_14ee71991758.slice/crio-f1e68110498a870c7bb06ff93a8b5c9d5efd7838f0cd3815763be8664fc6b132 WatchSource:0}: Error finding container f1e68110498a870c7bb06ff93a8b5c9d5efd7838f0cd3815763be8664fc6b132: Status 404 returned error can't find the container with id f1e68110498a870c7bb06ff93a8b5c9d5efd7838f0cd3815763be8664fc6b132
Jul 17 10:14:56.706004 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.705965    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 10:14:56.706111 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.706055    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 10:14:56.706111 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:56.706103    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 10:14:57.277131 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:57.277085    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl" event={"ID":"723a906f-455e-470a-bce6-14ee71991758","Type":"ContainerStarted","Data":"c1b3a271164f34d85bc1d7f323d5516e57325b7e7fcc3b29683069fcb5a7e89c"}
Jul 17 10:14:57.277131 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:57.277137    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl" event={"ID":"723a906f-455e-470a-bce6-14ee71991758","Type":"ContainerStarted","Data":"f1e68110498a870c7bb06ff93a8b5c9d5efd7838f0cd3815763be8664fc6b132"}
Jul 17 10:14:57.300263 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:57.300194    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl" podStartSLOduration=1.300179628 podStartE2EDuration="1.300179628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:14:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:14:57.296728085 +0000 UTC m=+2894.879366463" watchObservedRunningTime="2026-07-17 10:14:57.300179628 +0000 UTC m=+2894.882818004"
Jul 17 10:14:57.551332 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:57.551226    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:57.556191 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:57.556167    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:58.282863 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:58.282835    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:14:58.283926 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:14:58.283905    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:15:14.323413 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:15:14.323373    2574 patch_prober.go:28] interesting pod/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.138:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 17 10:15:14.323812 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:15:14.323453    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl" podUID="723a906f-455e-470a-bce6-14ee71991758" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.138:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 17 10:15:16.389929 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:15:16.389830    2574 generic.go:356] "Generic (PLEG): container finished" podID="723a906f-455e-470a-bce6-14ee71991758" containerID="c1b3a271164f34d85bc1d7f323d5516e57325b7e7fcc3b29683069fcb5a7e89c" exitCode=0
Jul 17 10:15:16.389929 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:15:16.389871    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl" event={"ID":"723a906f-455e-470a-bce6-14ee71991758","Type":"ContainerDied","Data":"c1b3a271164f34d85bc1d7f323d5516e57325b7e7fcc3b29683069fcb5a7e89c"}
Jul 17 10:15:16.390407 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:15:16.390270    2574 scope.go:117] "RemoveContainer" containerID="c1b3a271164f34d85bc1d7f323d5516e57325b7e7fcc3b29683069fcb5a7e89c"
Jul 17 10:15:16.391416 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:15:16.391390    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 10:15:16.391486 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:15:16.391460    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 10:15:16.391525 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:15:16.391495    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 10:15:17.397915 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:15:17.397874    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl" event={"ID":"723a906f-455e-470a-bce6-14ee71991758","Type":"ContainerStarted","Data":"6d41b64b14171121380fd722ba62cec9ea862edfd3588ae1201316f35f796a12"}
Jul 17 10:15:17.551000 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:15:17.550959    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:15:17.551233 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:15:17.551208    2574 patch_prober.go:28] interesting pod/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused" start-of-body=
Jul 17 10:15:17.551353 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:15:17.551288    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl" podUID="723a906f-455e-470a-bce6-14ee71991758" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused"
Jul 17 10:15:18.550976 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:15:18.550937    2574 patch_prober.go:28] interesting pod/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused" start-of-body=
Jul 17 10:15:18.551392 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:15:18.550996    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl" podUID="723a906f-455e-470a-bce6-14ee71991758" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused"
Jul 17 10:15:19.551583 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:15:19.551542    2574 patch_prober.go:28] interesting pod/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused" start-of-body=
Jul 17 10:15:19.552035 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:15:19.551617    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl" podUID="723a906f-455e-470a-bce6-14ee71991758" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused"
Jul 17 10:15:20.551345 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:15:20.551300    2574 patch_prober.go:28] interesting pod/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused" start-of-body=
Jul 17 10:15:20.551550 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:15:20.551366    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl" podUID="723a906f-455e-470a-bce6-14ee71991758" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.138:15021/healthz/ready\": dial tcp 10.132.0.138:15021: connect: connection refused"
Jul 17 10:15:21.555217 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:15:21.555187    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:15:21.555632 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:15:21.555465    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:15:21.556150 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:15:21.556135    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:16:03.736736 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:03.736694    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"]
Jul 17 10:16:03.737339 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:03.736904    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl" podUID="723a906f-455e-470a-bce6-14ee71991758" containerName="istio-proxy" containerID="cri-o://6d41b64b14171121380fd722ba62cec9ea862edfd3588ae1201316f35f796a12" gracePeriod=30
Jul 17 10:16:06.558699 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:06.558658    2574 patch_prober.go:28] interesting pod/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 17 10:16:06.559079 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:06.558716    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl" podUID="723a906f-455e-470a-bce6-14ee71991758" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 10:16:09.011835 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.011768    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:16:09.136907 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.136874    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/723a906f-455e-470a-bce6-14ee71991758-istio-token\") pod \"723a906f-455e-470a-bce6-14ee71991758\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") "
Jul 17 10:16:09.137104 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.136917    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-workload-certs\") pod \"723a906f-455e-470a-bce6-14ee71991758\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") "
Jul 17 10:16:09.137104 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.136941    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/723a906f-455e-470a-bce6-14ee71991758-istio-podinfo\") pod \"723a906f-455e-470a-bce6-14ee71991758\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") "
Jul 17 10:16:09.137104 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.136993    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-workload-socket\") pod \"723a906f-455e-470a-bce6-14ee71991758\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") "
Jul 17 10:16:09.137104 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.137013    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdrv8\" (UniqueName: \"kubernetes.io/projected/723a906f-455e-470a-bce6-14ee71991758-kube-api-access-bdrv8\") pod \"723a906f-455e-470a-bce6-14ee71991758\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") "
Jul 17 10:16:09.137104 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.137053    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/723a906f-455e-470a-bce6-14ee71991758-istiod-ca-cert\") pod \"723a906f-455e-470a-bce6-14ee71991758\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") "
Jul 17 10:16:09.137104 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.137080    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-istio-data\") pod \"723a906f-455e-470a-bce6-14ee71991758\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") "
Jul 17 10:16:09.137462 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.137114    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-istio-envoy\") pod \"723a906f-455e-470a-bce6-14ee71991758\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") "
Jul 17 10:16:09.137462 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.137141    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-credential-socket\") pod \"723a906f-455e-470a-bce6-14ee71991758\" (UID: \"723a906f-455e-470a-bce6-14ee71991758\") "
Jul 17 10:16:09.137462 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.137213    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "723a906f-455e-470a-bce6-14ee71991758" (UID: "723a906f-455e-470a-bce6-14ee71991758"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:16:09.137462 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.137346    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "723a906f-455e-470a-bce6-14ee71991758" (UID: "723a906f-455e-470a-bce6-14ee71991758"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:16:09.137684 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.137554    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "723a906f-455e-470a-bce6-14ee71991758" (UID: "723a906f-455e-470a-bce6-14ee71991758"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:16:09.137684 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.137575    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-workload-certs\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:16:09.137684 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.137594    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-workload-socket\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:16:09.137826 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.137707    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/723a906f-455e-470a-bce6-14ee71991758-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "723a906f-455e-470a-bce6-14ee71991758" (UID: "723a906f-455e-470a-bce6-14ee71991758"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:16:09.138120 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.138102    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-istio-data" (OuterVolumeSpecName: "istio-data") pod "723a906f-455e-470a-bce6-14ee71991758" (UID: "723a906f-455e-470a-bce6-14ee71991758"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:16:09.139934 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.139903    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/723a906f-455e-470a-bce6-14ee71991758-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "723a906f-455e-470a-bce6-14ee71991758" (UID: "723a906f-455e-470a-bce6-14ee71991758"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 10:16:09.139934 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.139932    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "723a906f-455e-470a-bce6-14ee71991758" (UID: "723a906f-455e-470a-bce6-14ee71991758"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:16:09.140083 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.139962    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/723a906f-455e-470a-bce6-14ee71991758-kube-api-access-bdrv8" (OuterVolumeSpecName: "kube-api-access-bdrv8") pod "723a906f-455e-470a-bce6-14ee71991758" (UID: "723a906f-455e-470a-bce6-14ee71991758"). InnerVolumeSpecName "kube-api-access-bdrv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:16:09.140083 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.139980    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/723a906f-455e-470a-bce6-14ee71991758-istio-token" (OuterVolumeSpecName: "istio-token") pod "723a906f-455e-470a-bce6-14ee71991758" (UID: "723a906f-455e-470a-bce6-14ee71991758"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:16:09.238235 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.238210    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdrv8\" (UniqueName: \"kubernetes.io/projected/723a906f-455e-470a-bce6-14ee71991758-kube-api-access-bdrv8\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:16:09.238235 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.238233    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/723a906f-455e-470a-bce6-14ee71991758-istiod-ca-cert\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:16:09.238522 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.238275    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-istio-data\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:16:09.238522 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.238289    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-istio-envoy\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:16:09.238522 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.238301    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/723a906f-455e-470a-bce6-14ee71991758-credential-socket\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:16:09.238522 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.238312    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/723a906f-455e-470a-bce6-14ee71991758-istio-token\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:16:09.238522 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.238325    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/723a906f-455e-470a-bce6-14ee71991758-istio-podinfo\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:16:09.735896 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.735859    2574 generic.go:356] "Generic (PLEG): container finished" podID="723a906f-455e-470a-bce6-14ee71991758" containerID="6d41b64b14171121380fd722ba62cec9ea862edfd3588ae1201316f35f796a12" exitCode=0
Jul 17 10:16:09.735896 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.735910    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"
Jul 17 10:16:09.736267 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.735947    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl" event={"ID":"723a906f-455e-470a-bce6-14ee71991758","Type":"ContainerDied","Data":"6d41b64b14171121380fd722ba62cec9ea862edfd3588ae1201316f35f796a12"}
Jul 17 10:16:09.736267 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.736005    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl" event={"ID":"723a906f-455e-470a-bce6-14ee71991758","Type":"ContainerDied","Data":"f1e68110498a870c7bb06ff93a8b5c9d5efd7838f0cd3815763be8664fc6b132"}
Jul 17 10:16:09.736267 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.736028    2574 scope.go:117] "RemoveContainer" containerID="6d41b64b14171121380fd722ba62cec9ea862edfd3588ae1201316f35f796a12"
Jul 17 10:16:09.746288 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.746270    2574 scope.go:117] "RemoveContainer" containerID="c1b3a271164f34d85bc1d7f323d5516e57325b7e7fcc3b29683069fcb5a7e89c"
Jul 17 10:16:09.754827 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.754805    2574 scope.go:117] "RemoveContainer" containerID="6d41b64b14171121380fd722ba62cec9ea862edfd3588ae1201316f35f796a12"
Jul 17 10:16:09.755114 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:16:09.755091    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d41b64b14171121380fd722ba62cec9ea862edfd3588ae1201316f35f796a12\": container with ID starting with 6d41b64b14171121380fd722ba62cec9ea862edfd3588ae1201316f35f796a12 not found: ID does not exist" containerID="6d41b64b14171121380fd722ba62cec9ea862edfd3588ae1201316f35f796a12"
Jul 17 10:16:09.755209 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.755120    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d41b64b14171121380fd722ba62cec9ea862edfd3588ae1201316f35f796a12"} err="failed to get container status \"6d41b64b14171121380fd722ba62cec9ea862edfd3588ae1201316f35f796a12\": rpc error: code = NotFound desc = could not find container \"6d41b64b14171121380fd722ba62cec9ea862edfd3588ae1201316f35f796a12\": container with ID starting with 6d41b64b14171121380fd722ba62cec9ea862edfd3588ae1201316f35f796a12 not found: ID does not exist"
Jul 17 10:16:09.755209 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.755137    2574 scope.go:117] "RemoveContainer" containerID="c1b3a271164f34d85bc1d7f323d5516e57325b7e7fcc3b29683069fcb5a7e89c"
Jul 17 10:16:09.755435 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:16:09.755420    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1b3a271164f34d85bc1d7f323d5516e57325b7e7fcc3b29683069fcb5a7e89c\": container with ID starting with c1b3a271164f34d85bc1d7f323d5516e57325b7e7fcc3b29683069fcb5a7e89c not found: ID does not exist" containerID="c1b3a271164f34d85bc1d7f323d5516e57325b7e7fcc3b29683069fcb5a7e89c"
Jul 17 10:16:09.755482 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.755441    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1b3a271164f34d85bc1d7f323d5516e57325b7e7fcc3b29683069fcb5a7e89c"} err="failed to get container status \"c1b3a271164f34d85bc1d7f323d5516e57325b7e7fcc3b29683069fcb5a7e89c\": rpc error: code = NotFound desc = could not find container \"c1b3a271164f34d85bc1d7f323d5516e57325b7e7fcc3b29683069fcb5a7e89c\": container with ID starting with c1b3a271164f34d85bc1d7f323d5516e57325b7e7fcc3b29683069fcb5a7e89c not found: ID does not exist"
Jul 17 10:16:09.760403 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.760376    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"]
Jul 17 10:16:09.762824 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:09.762799    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-1baf67-openshift-default-55cb89d8c-lj2pl"]
Jul 17 10:16:10.897945 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:10.897911    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="723a906f-455e-470a-bce6-14ee71991758" path="/var/lib/kubelet/pods/723a906f-455e-470a-bce6-14ee71991758/volumes"
Jul 17 10:16:12.591739 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.591703    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"]
Jul 17 10:16:12.592176 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.592162    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b0de510-d71b-4b55-8217-3c9b8428668b" containerName="istio-proxy"
Jul 17 10:16:12.592219 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.592177    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b0de510-d71b-4b55-8217-3c9b8428668b" containerName="istio-proxy"
Jul 17 10:16:12.592219 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.592197    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="723a906f-455e-470a-bce6-14ee71991758" containerName="istio-proxy"
Jul 17 10:16:12.592219 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.592203    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="723a906f-455e-470a-bce6-14ee71991758" containerName="istio-proxy"
Jul 17 10:16:12.592330 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.592296    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="723a906f-455e-470a-bce6-14ee71991758" containerName="istio-proxy"
Jul 17 10:16:12.592330 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.592309    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="723a906f-455e-470a-bce6-14ee71991758" containerName="istio-proxy"
Jul 17 10:16:12.597404 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.597385    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:12.601467 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.601445    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-af80d9-openshift-default-dockercfg-8f95m\""
Jul 17 10:16:12.611618 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.611591    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"]
Jul 17 10:16:12.667095 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.667059    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-istio-data\") pod \"e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") " pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:12.667095 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.667094    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-workload-certs\") pod \"e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") " pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:12.667316 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.667112    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf340eb4-e72a-4fe8-b125-158eb16e0543-istio-podinfo\") pod \"e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") " pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:12.667316 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.667189    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf340eb4-e72a-4fe8-b125-158eb16e0543-istiod-ca-cert\") pod \"e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") " pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:12.667316 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.667258    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-istio-envoy\") pod \"e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") " pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:12.667428 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.667343    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-credential-socket\") pod \"e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") " pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:12.667428 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.667362    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf340eb4-e72a-4fe8-b125-158eb16e0543-istio-token\") pod \"e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") " pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:12.667428 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.667409    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qllph\" (UniqueName: \"kubernetes.io/projected/cf340eb4-e72a-4fe8-b125-158eb16e0543-kube-api-access-qllph\") pod \"e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") " pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:12.667526 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.667484    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-workload-socket\") pod \"e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") " pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:12.768729 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.768696    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-credential-socket\") pod \"e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") " pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:12.768729 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.768730    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf340eb4-e72a-4fe8-b125-158eb16e0543-istio-token\") pod \"e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") " pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:12.768970 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.768768    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qllph\" (UniqueName: \"kubernetes.io/projected/cf340eb4-e72a-4fe8-b125-158eb16e0543-kube-api-access-qllph\") pod \"e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") " pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:12.768970 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.768789    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-workload-socket\") pod \"e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") " pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:12.768970 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.768812    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-istio-data\") pod \"e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") " pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:12.768970 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.768832    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-workload-certs\") pod \"e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") " pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:12.768970 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.768860    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf340eb4-e72a-4fe8-b125-158eb16e0543-istio-podinfo\") pod \"e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") " pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:12.768970 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.768909    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf340eb4-e72a-4fe8-b125-158eb16e0543-istiod-ca-cert\") pod \"e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") " pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:12.768970 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.768949    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-istio-envoy\") pod \"e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") " pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:12.769329 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.769087    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-credential-socket\") pod \"e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") " pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:12.769329 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.769173    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-workload-socket\") pod \"e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") " pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:12.769329 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.769229    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-istio-data\") pod \"e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") " pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:12.769472 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.769408    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-workload-certs\") pod \"e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") " pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:12.769712 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.769693    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf340eb4-e72a-4fe8-b125-158eb16e0543-istiod-ca-cert\") pod \"e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") " pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:12.771473 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.771442    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-istio-envoy\") pod \"e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") " pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:12.771594 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.771516    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf340eb4-e72a-4fe8-b125-158eb16e0543-istio-podinfo\") pod \"e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") " pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:12.777527 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.777505    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf340eb4-e72a-4fe8-b125-158eb16e0543-istio-token\") pod \"e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") " pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:12.777907 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.777886    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qllph\" (UniqueName: \"kubernetes.io/projected/cf340eb4-e72a-4fe8-b125-158eb16e0543-kube-api-access-qllph\") pod \"e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") " pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:12.909811 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:12.909762    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:13.063998 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:13.063968    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"]
Jul 17 10:16:13.065037 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:16:13.065009    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf340eb4_e72a_4fe8_b125_158eb16e0543.slice/crio-40c01f3f101af8b53bface510179e81ccad91da5a79b6f44f33df5af5ee9d43a WatchSource:0}: Error finding container 40c01f3f101af8b53bface510179e81ccad91da5a79b6f44f33df5af5ee9d43a: Status 404 returned error can't find the container with id 40c01f3f101af8b53bface510179e81ccad91da5a79b6f44f33df5af5ee9d43a
Jul 17 10:16:13.067685 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:13.067649    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 10:16:13.067774 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:13.067733    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 10:16:13.067814 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:13.067780    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 10:16:13.761768 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:13.761729    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp" event={"ID":"cf340eb4-e72a-4fe8-b125-158eb16e0543","Type":"ContainerStarted","Data":"fd4d15df6598ef507cab8c7870cc4442187ae80b71f56cafb78510e19306c226"}
Jul 17 10:16:13.762303 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:13.761803    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp" event={"ID":"cf340eb4-e72a-4fe8-b125-158eb16e0543","Type":"ContainerStarted","Data":"40c01f3f101af8b53bface510179e81ccad91da5a79b6f44f33df5af5ee9d43a"}
Jul 17 10:16:13.783419 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:13.783376    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp" podStartSLOduration=1.783360568 podStartE2EDuration="1.783360568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:16:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:16:13.781624241 +0000 UTC m=+2971.364262617" watchObservedRunningTime="2026-07-17 10:16:13.783360568 +0000 UTC m=+2971.365998944"
Jul 17 10:16:13.910273 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:13.910221    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:14.915957 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:14.915925    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:15.772789 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:15.772758    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:15.773870 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:15.773851    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:26.849083 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:26.848980    2574 generic.go:356] "Generic (PLEG): container finished" podID="cf340eb4-e72a-4fe8-b125-158eb16e0543" containerID="fd4d15df6598ef507cab8c7870cc4442187ae80b71f56cafb78510e19306c226" exitCode=0
Jul 17 10:16:26.849083 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:26.849055    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp" event={"ID":"cf340eb4-e72a-4fe8-b125-158eb16e0543","Type":"ContainerDied","Data":"fd4d15df6598ef507cab8c7870cc4442187ae80b71f56cafb78510e19306c226"}
Jul 17 10:16:26.849567 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:26.849487    2574 scope.go:117] "RemoveContainer" containerID="fd4d15df6598ef507cab8c7870cc4442187ae80b71f56cafb78510e19306c226"
Jul 17 10:16:26.850546 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:26.850520    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 10:16:26.850628 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:26.850591    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 10:16:26.850628 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:26.850624    2574 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 17 10:16:27.857722 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:27.857680    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp" event={"ID":"cf340eb4-e72a-4fe8-b125-158eb16e0543","Type":"ContainerStarted","Data":"10d0d7feda4aa2787e0cbf957c81051939ba0a7f11214784853e39b11d2620aa"}
Jul 17 10:16:27.910517 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:27.910416    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:27.911372 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:27.911338    2574 patch_prober.go:28] interesting pod/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused" start-of-body=
Jul 17 10:16:27.911525 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:27.911399    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp" podUID="cf340eb4-e72a-4fe8-b125-158eb16e0543" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused"
Jul 17 10:16:28.910799 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:28.910757    2574 patch_prober.go:28] interesting pod/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused" start-of-body=
Jul 17 10:16:28.911324 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:28.910831    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp" podUID="cf340eb4-e72a-4fe8-b125-158eb16e0543" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused"
Jul 17 10:16:29.910738 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:29.910696    2574 patch_prober.go:28] interesting pod/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused" start-of-body=
Jul 17 10:16:29.911063 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:29.911029    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp" podUID="cf340eb4-e72a-4fe8-b125-158eb16e0543" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused"
Jul 17 10:16:30.772963 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:30.772932    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:30.910525 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:30.910482    2574 patch_prober.go:28] interesting pod/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused" start-of-body=
Jul 17 10:16:30.910751 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:30.910554    2574 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp" podUID="cf340eb4-e72a-4fe8-b125-158eb16e0543" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.139:15021/healthz/ready\": dial tcp 10.132.0.139:15021: connect: connection refused"
Jul 17 10:16:31.915258 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:31.915204    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:31.916298 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:31.916277    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:16:43.174353 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:43.174322    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dh564_84db06ab-87e2-43b3-82d0-13f06ba86d06/ovn-acl-logging/0.log"
Jul 17 10:16:43.180799 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:16:43.180773    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dh564_84db06ab-87e2-43b3-82d0-13f06ba86d06/ovn-acl-logging/0.log"
Jul 17 10:17:26.562981 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:26.562929    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb"]
Jul 17 10:17:26.563718 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:26.563695    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="723a906f-455e-470a-bce6-14ee71991758" containerName="istio-proxy"
Jul 17 10:17:26.563812 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:26.563722    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="723a906f-455e-470a-bce6-14ee71991758" containerName="istio-proxy"
Jul 17 10:17:26.566096 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:26.566074    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb"
Jul 17 10:17:26.568804 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:26.568784    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:17:26.573378 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:26.573351    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb"]
Jul 17 10:17:26.632863 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:26.632830    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/853e8f6b-1810-49dd-9d9f-61c96bc1acf7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb\" (UID: \"853e8f6b-1810-49dd-9d9f-61c96bc1acf7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb"
Jul 17 10:17:26.733593 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:26.733558    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/853e8f6b-1810-49dd-9d9f-61c96bc1acf7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb\" (UID: \"853e8f6b-1810-49dd-9d9f-61c96bc1acf7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb"
Jul 17 10:17:26.734204 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:26.734183    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/853e8f6b-1810-49dd-9d9f-61c96bc1acf7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb\" (UID: \"853e8f6b-1810-49dd-9d9f-61c96bc1acf7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb"
Jul 17 10:17:26.879117 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:26.879027    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb"
Jul 17 10:17:27.024805 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:27.024777    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb"]
Jul 17 10:17:27.026480 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:17:27.026448    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod853e8f6b_1810_49dd_9d9f_61c96bc1acf7.slice/crio-278b08357320a33ee497bb621df70ae0374471013c430658a0f7efa8903fb565 WatchSource:0}: Error finding container 278b08357320a33ee497bb621df70ae0374471013c430658a0f7efa8903fb565: Status 404 returned error can't find the container with id 278b08357320a33ee497bb621df70ae0374471013c430658a0f7efa8903fb565
Jul 17 10:17:27.199058 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:27.199012    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb" event={"ID":"853e8f6b-1810-49dd-9d9f-61c96bc1acf7","Type":"ContainerStarted","Data":"278b08357320a33ee497bb621df70ae0374471013c430658a0f7efa8903fb565"}
Jul 17 10:17:28.210218 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:28.210184    2574 generic.go:356] "Generic (PLEG): container finished" podID="853e8f6b-1810-49dd-9d9f-61c96bc1acf7" containerID="1ecf8469e066752b6171951c8234d8699f7ffa8db0f47069bb49529d5b6506a8" exitCode=6
Jul 17 10:17:28.210641 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:28.210276    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb" event={"ID":"853e8f6b-1810-49dd-9d9f-61c96bc1acf7","Type":"ContainerDied","Data":"1ecf8469e066752b6171951c8234d8699f7ffa8db0f47069bb49529d5b6506a8"}
Jul 17 10:17:28.210641 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:28.210483    2574 scope.go:117] "RemoveContainer" containerID="1ecf8469e066752b6171951c8234d8699f7ffa8db0f47069bb49529d5b6506a8"
Jul 17 10:17:29.217901 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:29.217869    2574 generic.go:356] "Generic (PLEG): container finished" podID="853e8f6b-1810-49dd-9d9f-61c96bc1acf7" containerID="6e4e562895c4c26691b97ae06342ac8ea68b1bd1de22ddab3017e61ff6cf5f2a" exitCode=6
Jul 17 10:17:29.218460 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:29.217962    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb" event={"ID":"853e8f6b-1810-49dd-9d9f-61c96bc1acf7","Type":"ContainerDied","Data":"6e4e562895c4c26691b97ae06342ac8ea68b1bd1de22ddab3017e61ff6cf5f2a"}
Jul 17 10:17:29.218460 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:29.218035    2574 scope.go:117] "RemoveContainer" containerID="1ecf8469e066752b6171951c8234d8699f7ffa8db0f47069bb49529d5b6506a8"
Jul 17 10:17:29.218460 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:29.218169    2574 scope.go:117] "RemoveContainer" containerID="6e4e562895c4c26691b97ae06342ac8ea68b1bd1de22ddab3017e61ff6cf5f2a"
Jul 17 10:17:29.218593 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:17:29.218480    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb_odh-ai-gateway-infra(853e8f6b-1810-49dd-9d9f-61c96bc1acf7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb" podUID="853e8f6b-1810-49dd-9d9f-61c96bc1acf7"
Jul 17 10:17:30.226455 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:30.226429    2574 scope.go:117] "RemoveContainer" containerID="6e4e562895c4c26691b97ae06342ac8ea68b1bd1de22ddab3017e61ff6cf5f2a"
Jul 17 10:17:30.226828 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:17:30.226666    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb_odh-ai-gateway-infra(853e8f6b-1810-49dd-9d9f-61c96bc1acf7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb" podUID="853e8f6b-1810-49dd-9d9f-61c96bc1acf7"
Jul 17 10:17:37.793919 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:37.793831    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9"]
Jul 17 10:17:37.797688 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:37.797667    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9"
Jul 17 10:17:37.807350 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:37.807314    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9"]
Jul 17 10:17:37.966819 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:37.966778    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53d5285b-8a3d-4932-8e57-5143bacafeae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9\" (UID: \"53d5285b-8a3d-4932-8e57-5143bacafeae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9"
Jul 17 10:17:38.067991 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:38.067875    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53d5285b-8a3d-4932-8e57-5143bacafeae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9\" (UID: \"53d5285b-8a3d-4932-8e57-5143bacafeae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9"
Jul 17 10:17:38.068554 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:38.068530    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53d5285b-8a3d-4932-8e57-5143bacafeae-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9\" (UID: \"53d5285b-8a3d-4932-8e57-5143bacafeae\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9"
Jul 17 10:17:38.111626 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:38.111587    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9"
Jul 17 10:17:38.259289 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:38.259259    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9"]
Jul 17 10:17:38.262212 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:17:38.262174    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53d5285b_8a3d_4932_8e57_5143bacafeae.slice/crio-b7a219e91ef189b587ea1e1e2e6fbf55596bd0fef4e41fc310d76ef161077a2a WatchSource:0}: Error finding container b7a219e91ef189b587ea1e1e2e6fbf55596bd0fef4e41fc310d76ef161077a2a: Status 404 returned error can't find the container with id b7a219e91ef189b587ea1e1e2e6fbf55596bd0fef4e41fc310d76ef161077a2a
Jul 17 10:17:38.272070 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:38.272035    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9" event={"ID":"53d5285b-8a3d-4932-8e57-5143bacafeae","Type":"ContainerStarted","Data":"b7a219e91ef189b587ea1e1e2e6fbf55596bd0fef4e41fc310d76ef161077a2a"}
Jul 17 10:17:39.279256 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:39.279154    2574 generic.go:356] "Generic (PLEG): container finished" podID="53d5285b-8a3d-4932-8e57-5143bacafeae" containerID="da7aea0505860f656a09c6ed675e594db4315dd48ba1bdf57dc4f9f80cfcaa9c" exitCode=6
Jul 17 10:17:39.279636 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:39.279259    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9" event={"ID":"53d5285b-8a3d-4932-8e57-5143bacafeae","Type":"ContainerDied","Data":"da7aea0505860f656a09c6ed675e594db4315dd48ba1bdf57dc4f9f80cfcaa9c"}
Jul 17 10:17:39.279636 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:39.279436    2574 scope.go:117] "RemoveContainer" containerID="da7aea0505860f656a09c6ed675e594db4315dd48ba1bdf57dc4f9f80cfcaa9c"
Jul 17 10:17:40.287067 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:40.287036    2574 generic.go:356] "Generic (PLEG): container finished" podID="53d5285b-8a3d-4932-8e57-5143bacafeae" containerID="7d53bbabe06d278df8afef3ec2bb4ad997c7467c3b3b9d0cfa3ce3c3c0409f82" exitCode=6
Jul 17 10:17:40.287586 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:40.287118    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9" event={"ID":"53d5285b-8a3d-4932-8e57-5143bacafeae","Type":"ContainerDied","Data":"7d53bbabe06d278df8afef3ec2bb4ad997c7467c3b3b9d0cfa3ce3c3c0409f82"}
Jul 17 10:17:40.287586 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:40.287179    2574 scope.go:117] "RemoveContainer" containerID="da7aea0505860f656a09c6ed675e594db4315dd48ba1bdf57dc4f9f80cfcaa9c"
Jul 17 10:17:40.287586 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:40.287425    2574 scope.go:117] "RemoveContainer" containerID="7d53bbabe06d278df8afef3ec2bb4ad997c7467c3b3b9d0cfa3ce3c3c0409f82"
Jul 17 10:17:40.287753 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:17:40.287700    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9_odh-ai-gateway-infra(53d5285b-8a3d-4932-8e57-5143bacafeae)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9" podUID="53d5285b-8a3d-4932-8e57-5143bacafeae"
Jul 17 10:17:41.294775 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:41.294748    2574 scope.go:117] "RemoveContainer" containerID="7d53bbabe06d278df8afef3ec2bb4ad997c7467c3b3b9d0cfa3ce3c3c0409f82"
Jul 17 10:17:41.295194 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:17:41.294969    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9_odh-ai-gateway-infra(53d5285b-8a3d-4932-8e57-5143bacafeae)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9" podUID="53d5285b-8a3d-4932-8e57-5143bacafeae"
Jul 17 10:17:42.895907 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:42.895875    2574 scope.go:117] "RemoveContainer" containerID="6e4e562895c4c26691b97ae06342ac8ea68b1bd1de22ddab3017e61ff6cf5f2a"
Jul 17 10:17:43.309373 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:43.309337    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb" event={"ID":"853e8f6b-1810-49dd-9d9f-61c96bc1acf7","Type":"ContainerStarted","Data":"e19e9543589823d6b8a9d6a7e5f012551ac0e2f789d338e6e1e92dca9e8a76be"}
Jul 17 10:17:43.326756 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:43.326693    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb" podStartSLOduration=17.326676926 podStartE2EDuration="17.326676926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:17:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:17:43.325298741 +0000 UTC m=+3060.907937116" watchObservedRunningTime="2026-07-17 10:17:43.326676926 +0000 UTC m=+3060.909315299"
Jul 17 10:17:44.316541 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:44.316504    2574 generic.go:356] "Generic (PLEG): container finished" podID="853e8f6b-1810-49dd-9d9f-61c96bc1acf7" containerID="e19e9543589823d6b8a9d6a7e5f012551ac0e2f789d338e6e1e92dca9e8a76be" exitCode=6
Jul 17 10:17:44.316952 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:44.316578    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb" event={"ID":"853e8f6b-1810-49dd-9d9f-61c96bc1acf7","Type":"ContainerDied","Data":"e19e9543589823d6b8a9d6a7e5f012551ac0e2f789d338e6e1e92dca9e8a76be"}
Jul 17 10:17:44.316952 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:44.316646    2574 scope.go:117] "RemoveContainer" containerID="6e4e562895c4c26691b97ae06342ac8ea68b1bd1de22ddab3017e61ff6cf5f2a"
Jul 17 10:17:44.317064 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:44.317009    2574 scope.go:117] "RemoveContainer" containerID="e19e9543589823d6b8a9d6a7e5f012551ac0e2f789d338e6e1e92dca9e8a76be"
Jul 17 10:17:44.317280 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:17:44.317232    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb_odh-ai-gateway-infra(853e8f6b-1810-49dd-9d9f-61c96bc1acf7)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb" podUID="853e8f6b-1810-49dd-9d9f-61c96bc1acf7"
Jul 17 10:17:44.338867 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:44.338835    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb"]
Jul 17 10:17:45.453806 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:45.453776    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb"
Jul 17 10:17:45.537161 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:45.537123    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/853e8f6b-1810-49dd-9d9f-61c96bc1acf7-maas-api-service-ca\") pod \"853e8f6b-1810-49dd-9d9f-61c96bc1acf7\" (UID: \"853e8f6b-1810-49dd-9d9f-61c96bc1acf7\") "
Jul 17 10:17:45.537465 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:45.537442    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/853e8f6b-1810-49dd-9d9f-61c96bc1acf7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "853e8f6b-1810-49dd-9d9f-61c96bc1acf7" (UID: "853e8f6b-1810-49dd-9d9f-61c96bc1acf7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:17:45.638388 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:45.638300    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/853e8f6b-1810-49dd-9d9f-61c96bc1acf7-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:17:46.336979 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:46.336937    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb" event={"ID":"853e8f6b-1810-49dd-9d9f-61c96bc1acf7","Type":"ContainerDied","Data":"278b08357320a33ee497bb621df70ae0374471013c430658a0f7efa8903fb565"}
Jul 17 10:17:46.337325 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:46.337301    2574 scope.go:117] "RemoveContainer" containerID="e19e9543589823d6b8a9d6a7e5f012551ac0e2f789d338e6e1e92dca9e8a76be"
Jul 17 10:17:46.337581 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:46.337561    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb"
Jul 17 10:17:46.363081 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:46.363053    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb"]
Jul 17 10:17:46.365276 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:46.365255    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-b4nxb"]
Jul 17 10:17:46.897835 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:46.897790    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="853e8f6b-1810-49dd-9d9f-61c96bc1acf7" path="/var/lib/kubelet/pods/853e8f6b-1810-49dd-9d9f-61c96bc1acf7/volumes"
Jul 17 10:17:51.891861 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:51.891826    2574 scope.go:117] "RemoveContainer" containerID="7d53bbabe06d278df8afef3ec2bb4ad997c7467c3b3b9d0cfa3ce3c3c0409f82"
Jul 17 10:17:52.376266 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:52.376198    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9" event={"ID":"53d5285b-8a3d-4932-8e57-5143bacafeae","Type":"ContainerStarted","Data":"4e6173e512848ae8547c8ecbea8a450537657abca7760dd2e5889e7c59a28b41"}
Jul 17 10:17:52.394905 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:52.394844    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9" podStartSLOduration=15.394823924 podStartE2EDuration="15.394823924s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:17:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:17:52.39275537 +0000 UTC m=+3069.975393746" watchObservedRunningTime="2026-07-17 10:17:52.394823924 +0000 UTC m=+3069.977462299"
Jul 17 10:17:53.386487 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:53.386447    2574 generic.go:356] "Generic (PLEG): container finished" podID="53d5285b-8a3d-4932-8e57-5143bacafeae" containerID="4e6173e512848ae8547c8ecbea8a450537657abca7760dd2e5889e7c59a28b41" exitCode=6
Jul 17 10:17:53.386944 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:53.386509    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9" event={"ID":"53d5285b-8a3d-4932-8e57-5143bacafeae","Type":"ContainerDied","Data":"4e6173e512848ae8547c8ecbea8a450537657abca7760dd2e5889e7c59a28b41"}
Jul 17 10:17:53.386944 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:53.386578    2574 scope.go:117] "RemoveContainer" containerID="7d53bbabe06d278df8afef3ec2bb4ad997c7467c3b3b9d0cfa3ce3c3c0409f82"
Jul 17 10:17:53.387056 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:53.386952    2574 scope.go:117] "RemoveContainer" containerID="4e6173e512848ae8547c8ecbea8a450537657abca7760dd2e5889e7c59a28b41"
Jul 17 10:17:53.387232 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:17:53.387210    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9_odh-ai-gateway-infra(53d5285b-8a3d-4932-8e57-5143bacafeae)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9" podUID="53d5285b-8a3d-4932-8e57-5143bacafeae"
Jul 17 10:17:53.403820 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:53.403775    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9"]
Jul 17 10:17:54.553685 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:54.553653    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9"
Jul 17 10:17:54.625310 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:54.625232    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53d5285b-8a3d-4932-8e57-5143bacafeae-maas-api-service-ca\") pod \"53d5285b-8a3d-4932-8e57-5143bacafeae\" (UID: \"53d5285b-8a3d-4932-8e57-5143bacafeae\") "
Jul 17 10:17:54.625599 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:54.625577    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53d5285b-8a3d-4932-8e57-5143bacafeae-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "53d5285b-8a3d-4932-8e57-5143bacafeae" (UID: "53d5285b-8a3d-4932-8e57-5143bacafeae"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:17:54.726396 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:54.726352    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53d5285b-8a3d-4932-8e57-5143bacafeae-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:17:55.402209 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:55.402171    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9" event={"ID":"53d5285b-8a3d-4932-8e57-5143bacafeae","Type":"ContainerDied","Data":"b7a219e91ef189b587ea1e1e2e6fbf55596bd0fef4e41fc310d76ef161077a2a"}
Jul 17 10:17:55.402459 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:55.402234    2574 scope.go:117] "RemoveContainer" containerID="4e6173e512848ae8547c8ecbea8a450537657abca7760dd2e5889e7c59a28b41"
Jul 17 10:17:55.402459 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:55.402185    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9"
Jul 17 10:17:55.426968 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:55.426934    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9"]
Jul 17 10:17:55.436094 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:55.436049    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-lgnr9"]
Jul 17 10:17:56.070333 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:56.070291    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-g27p9"]
Jul 17 10:17:56.070952 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:56.070931    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53d5285b-8a3d-4932-8e57-5143bacafeae" containerName="revoke-keys"
Jul 17 10:17:56.071046 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:56.070955    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="53d5285b-8a3d-4932-8e57-5143bacafeae" containerName="revoke-keys"
Jul 17 10:17:56.071046 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:56.070971    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53d5285b-8a3d-4932-8e57-5143bacafeae" containerName="revoke-keys"
Jul 17 10:17:56.071046 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:56.070980    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="53d5285b-8a3d-4932-8e57-5143bacafeae" containerName="revoke-keys"
Jul 17 10:17:56.071046 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:56.071001    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="853e8f6b-1810-49dd-9d9f-61c96bc1acf7" containerName="revoke-keys"
Jul 17 10:17:56.071046 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:56.071010    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="853e8f6b-1810-49dd-9d9f-61c96bc1acf7" containerName="revoke-keys"
Jul 17 10:17:56.071046 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:56.071023    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="853e8f6b-1810-49dd-9d9f-61c96bc1acf7" containerName="revoke-keys"
Jul 17 10:17:56.071046 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:56.071031    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="853e8f6b-1810-49dd-9d9f-61c96bc1acf7" containerName="revoke-keys"
Jul 17 10:17:56.071434 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:56.071055    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="853e8f6b-1810-49dd-9d9f-61c96bc1acf7" containerName="revoke-keys"
Jul 17 10:17:56.071434 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:56.071064    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="853e8f6b-1810-49dd-9d9f-61c96bc1acf7" containerName="revoke-keys"
Jul 17 10:17:56.071434 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:56.071173    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="53d5285b-8a3d-4932-8e57-5143bacafeae" containerName="revoke-keys"
Jul 17 10:17:56.071434 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:56.071185    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="853e8f6b-1810-49dd-9d9f-61c96bc1acf7" containerName="revoke-keys"
Jul 17 10:17:56.071434 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:56.071194    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="53d5285b-8a3d-4932-8e57-5143bacafeae" containerName="revoke-keys"
Jul 17 10:17:56.071434 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:56.071207    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="853e8f6b-1810-49dd-9d9f-61c96bc1acf7" containerName="revoke-keys"
Jul 17 10:17:56.075898 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:56.075867    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-g27p9"
Jul 17 10:17:56.079167 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:56.079122    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:17:56.081983 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:56.081954    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-g27p9"]
Jul 17 10:17:56.141090 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:56.141051    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cc082e2-cebe-49e8-9f30-45f983a493cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-g27p9\" (UID: \"4cc082e2-cebe-49e8-9f30-45f983a493cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-g27p9"
Jul 17 10:17:56.242642 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:56.242593    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cc082e2-cebe-49e8-9f30-45f983a493cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-g27p9\" (UID: \"4cc082e2-cebe-49e8-9f30-45f983a493cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-g27p9"
Jul 17 10:17:56.243469 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:56.243446    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cc082e2-cebe-49e8-9f30-45f983a493cb-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-g27p9\" (UID: \"4cc082e2-cebe-49e8-9f30-45f983a493cb\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-g27p9"
Jul 17 10:17:56.389208 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:56.389104    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-g27p9"
Jul 17 10:17:56.541222 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:56.541191    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-g27p9"]
Jul 17 10:17:56.543088 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:17:56.543036    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cc082e2_cebe_49e8_9f30_45f983a493cb.slice/crio-fcbd69e51cdee9100ff077203e205fd3efc32d8346374238c7108cfec7d4ae72 WatchSource:0}: Error finding container fcbd69e51cdee9100ff077203e205fd3efc32d8346374238c7108cfec7d4ae72: Status 404 returned error can't find the container with id fcbd69e51cdee9100ff077203e205fd3efc32d8346374238c7108cfec7d4ae72
Jul 17 10:17:56.897660 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:56.897622    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53d5285b-8a3d-4932-8e57-5143bacafeae" path="/var/lib/kubelet/pods/53d5285b-8a3d-4932-8e57-5143bacafeae/volumes"
Jul 17 10:17:57.425402 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:57.425364    2574 generic.go:356] "Generic (PLEG): container finished" podID="4cc082e2-cebe-49e8-9f30-45f983a493cb" containerID="1fc624a8c2e59881594af3f7f53ca7730d4721abf0e820ac4065340c92fc302d" exitCode=6
Jul 17 10:17:57.425893 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:57.425448    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-g27p9" event={"ID":"4cc082e2-cebe-49e8-9f30-45f983a493cb","Type":"ContainerDied","Data":"1fc624a8c2e59881594af3f7f53ca7730d4721abf0e820ac4065340c92fc302d"}
Jul 17 10:17:57.425893 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:57.425534    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-g27p9" event={"ID":"4cc082e2-cebe-49e8-9f30-45f983a493cb","Type":"ContainerStarted","Data":"fcbd69e51cdee9100ff077203e205fd3efc32d8346374238c7108cfec7d4ae72"}
Jul 17 10:17:57.425893 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:57.425649    2574 scope.go:117] "RemoveContainer" containerID="1fc624a8c2e59881594af3f7f53ca7730d4721abf0e820ac4065340c92fc302d"
Jul 17 10:17:58.433997 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:58.433957    2574 generic.go:356] "Generic (PLEG): container finished" podID="4cc082e2-cebe-49e8-9f30-45f983a493cb" containerID="f3f34d9e240de8d35df48055d3d90fd33554bc7054bd33ea6f5e83ddde596c39" exitCode=6
Jul 17 10:17:58.434507 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:58.434043    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-g27p9" event={"ID":"4cc082e2-cebe-49e8-9f30-45f983a493cb","Type":"ContainerDied","Data":"f3f34d9e240de8d35df48055d3d90fd33554bc7054bd33ea6f5e83ddde596c39"}
Jul 17 10:17:58.434507 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:58.434107    2574 scope.go:117] "RemoveContainer" containerID="1fc624a8c2e59881594af3f7f53ca7730d4721abf0e820ac4065340c92fc302d"
Jul 17 10:17:58.434507 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:58.434343    2574 scope.go:117] "RemoveContainer" containerID="f3f34d9e240de8d35df48055d3d90fd33554bc7054bd33ea6f5e83ddde596c39"
Jul 17 10:17:58.434686 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:17:58.434635    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-g27p9_odh-ai-gateway-infra(4cc082e2-cebe-49e8-9f30-45f983a493cb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-g27p9" podUID="4cc082e2-cebe-49e8-9f30-45f983a493cb"
Jul 17 10:17:58.735913 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:58.735807    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn"]
Jul 17 10:17:58.736407 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:58.736386    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53d5285b-8a3d-4932-8e57-5143bacafeae" containerName="revoke-keys"
Jul 17 10:17:58.736503 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:58.736408    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="53d5285b-8a3d-4932-8e57-5143bacafeae" containerName="revoke-keys"
Jul 17 10:17:58.736569 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:58.736529    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="853e8f6b-1810-49dd-9d9f-61c96bc1acf7" containerName="revoke-keys"
Jul 17 10:17:58.736569 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:58.736541    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="53d5285b-8a3d-4932-8e57-5143bacafeae" containerName="revoke-keys"
Jul 17 10:17:58.740871 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:58.740846    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn"
Jul 17 10:17:58.746330 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:58.746296    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn"]
Jul 17 10:17:58.869781 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:58.869731    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/498055ea-fdc2-4a1e-8bba-6f92d8675e39-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn\" (UID: \"498055ea-fdc2-4a1e-8bba-6f92d8675e39\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn"
Jul 17 10:17:58.971389 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:58.971352    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/498055ea-fdc2-4a1e-8bba-6f92d8675e39-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn\" (UID: \"498055ea-fdc2-4a1e-8bba-6f92d8675e39\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn"
Jul 17 10:17:58.972064 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:58.972036    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/498055ea-fdc2-4a1e-8bba-6f92d8675e39-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn\" (UID: \"498055ea-fdc2-4a1e-8bba-6f92d8675e39\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn"
Jul 17 10:17:59.055562 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:59.055454    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn"
Jul 17 10:17:59.212753 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:59.212716    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn"]
Jul 17 10:17:59.213710 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:17:59.213681    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod498055ea_fdc2_4a1e_8bba_6f92d8675e39.slice/crio-78ccbcee00d0980696fba62a4abc8721ba1e033198f2586fbceabedc3f2085ae WatchSource:0}: Error finding container 78ccbcee00d0980696fba62a4abc8721ba1e033198f2586fbceabedc3f2085ae: Status 404 returned error can't find the container with id 78ccbcee00d0980696fba62a4abc8721ba1e033198f2586fbceabedc3f2085ae
Jul 17 10:17:59.443694 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:59.443660    2574 scope.go:117] "RemoveContainer" containerID="f3f34d9e240de8d35df48055d3d90fd33554bc7054bd33ea6f5e83ddde596c39"
Jul 17 10:17:59.444146 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:17:59.443970    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-g27p9_odh-ai-gateway-infra(4cc082e2-cebe-49e8-9f30-45f983a493cb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-g27p9" podUID="4cc082e2-cebe-49e8-9f30-45f983a493cb"
Jul 17 10:17:59.449316 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:17:59.449278    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn" event={"ID":"498055ea-fdc2-4a1e-8bba-6f92d8675e39","Type":"ContainerStarted","Data":"78ccbcee00d0980696fba62a4abc8721ba1e033198f2586fbceabedc3f2085ae"}
Jul 17 10:18:00.457845 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:00.457791    2574 generic.go:356] "Generic (PLEG): container finished" podID="498055ea-fdc2-4a1e-8bba-6f92d8675e39" containerID="94b969ad16905d5a8b24758a1f68ce10a8ba0588728f8020dc4d04ed4d75ece7" exitCode=6
Jul 17 10:18:00.458293 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:00.457881    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn" event={"ID":"498055ea-fdc2-4a1e-8bba-6f92d8675e39","Type":"ContainerDied","Data":"94b969ad16905d5a8b24758a1f68ce10a8ba0588728f8020dc4d04ed4d75ece7"}
Jul 17 10:18:00.458293 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:00.458135    2574 scope.go:117] "RemoveContainer" containerID="94b969ad16905d5a8b24758a1f68ce10a8ba0588728f8020dc4d04ed4d75ece7"
Jul 17 10:18:01.466110 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:01.466072    2574 generic.go:356] "Generic (PLEG): container finished" podID="498055ea-fdc2-4a1e-8bba-6f92d8675e39" containerID="e8120f3f6580f84c641eeb5e203de9933ba023896c5d1fba15c94925a8bb1cea" exitCode=6
Jul 17 10:18:01.466551 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:01.466143    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn" event={"ID":"498055ea-fdc2-4a1e-8bba-6f92d8675e39","Type":"ContainerDied","Data":"e8120f3f6580f84c641eeb5e203de9933ba023896c5d1fba15c94925a8bb1cea"}
Jul 17 10:18:01.466551 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:01.466225    2574 scope.go:117] "RemoveContainer" containerID="94b969ad16905d5a8b24758a1f68ce10a8ba0588728f8020dc4d04ed4d75ece7"
Jul 17 10:18:01.466551 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:01.466401    2574 scope.go:117] "RemoveContainer" containerID="e8120f3f6580f84c641eeb5e203de9933ba023896c5d1fba15c94925a8bb1cea"
Jul 17 10:18:01.466709 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:18:01.466688    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn_odh-ai-gateway-infra(498055ea-fdc2-4a1e-8bba-6f92d8675e39)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn" podUID="498055ea-fdc2-4a1e-8bba-6f92d8675e39"
Jul 17 10:18:02.474098 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:02.474070    2574 scope.go:117] "RemoveContainer" containerID="e8120f3f6580f84c641eeb5e203de9933ba023896c5d1fba15c94925a8bb1cea"
Jul 17 10:18:02.474562 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:18:02.474357    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn_odh-ai-gateway-infra(498055ea-fdc2-4a1e-8bba-6f92d8675e39)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn" podUID="498055ea-fdc2-4a1e-8bba-6f92d8675e39"
Jul 17 10:18:09.892195 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:09.892161    2574 scope.go:117] "RemoveContainer" containerID="f3f34d9e240de8d35df48055d3d90fd33554bc7054bd33ea6f5e83ddde596c39"
Jul 17 10:18:10.523774 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:10.523686    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-g27p9" event={"ID":"4cc082e2-cebe-49e8-9f30-45f983a493cb","Type":"ContainerStarted","Data":"65d5582c64dc812399a4d7b6c2e4163491be03a4bab711e596d48efc535e2f61"}
Jul 17 10:18:10.542291 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:10.542212    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-g27p9" podStartSLOduration=14.542191257 podStartE2EDuration="14.542191257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:17:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:18:10.54040613 +0000 UTC m=+3088.123044504" watchObservedRunningTime="2026-07-17 10:18:10.542191257 +0000 UTC m=+3088.124829633"
Jul 17 10:18:11.531926 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:11.531891    2574 generic.go:356] "Generic (PLEG): container finished" podID="4cc082e2-cebe-49e8-9f30-45f983a493cb" containerID="65d5582c64dc812399a4d7b6c2e4163491be03a4bab711e596d48efc535e2f61" exitCode=6
Jul 17 10:18:11.532392 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:11.531962    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-g27p9" event={"ID":"4cc082e2-cebe-49e8-9f30-45f983a493cb","Type":"ContainerDied","Data":"65d5582c64dc812399a4d7b6c2e4163491be03a4bab711e596d48efc535e2f61"}
Jul 17 10:18:11.532392 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:11.532040    2574 scope.go:117] "RemoveContainer" containerID="f3f34d9e240de8d35df48055d3d90fd33554bc7054bd33ea6f5e83ddde596c39"
Jul 17 10:18:11.532487 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:11.532394    2574 scope.go:117] "RemoveContainer" containerID="65d5582c64dc812399a4d7b6c2e4163491be03a4bab711e596d48efc535e2f61"
Jul 17 10:18:11.532702 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:18:11.532678    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-g27p9_odh-ai-gateway-infra(4cc082e2-cebe-49e8-9f30-45f983a493cb)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-g27p9" podUID="4cc082e2-cebe-49e8-9f30-45f983a493cb"
Jul 17 10:18:11.560001 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:11.559959    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-g27p9"]
Jul 17 10:18:12.694696 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:12.694660    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-g27p9"
Jul 17 10:18:12.817065 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:12.816977    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cc082e2-cebe-49e8-9f30-45f983a493cb-maas-api-service-ca\") pod \"4cc082e2-cebe-49e8-9f30-45f983a493cb\" (UID: \"4cc082e2-cebe-49e8-9f30-45f983a493cb\") "
Jul 17 10:18:12.817352 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:12.817328    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cc082e2-cebe-49e8-9f30-45f983a493cb-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4cc082e2-cebe-49e8-9f30-45f983a493cb" (UID: "4cc082e2-cebe-49e8-9f30-45f983a493cb"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:18:12.918435 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:12.918396    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cc082e2-cebe-49e8-9f30-45f983a493cb-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:18:13.168011 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:13.167974    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vnkn9"]
Jul 17 10:18:13.171432 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:13.168564    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cc082e2-cebe-49e8-9f30-45f983a493cb" containerName="revoke-keys"
Jul 17 10:18:13.171432 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:13.168601    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cc082e2-cebe-49e8-9f30-45f983a493cb" containerName="revoke-keys"
Jul 17 10:18:13.171432 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:13.168619    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cc082e2-cebe-49e8-9f30-45f983a493cb" containerName="revoke-keys"
Jul 17 10:18:13.171432 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:13.168628    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cc082e2-cebe-49e8-9f30-45f983a493cb" containerName="revoke-keys"
Jul 17 10:18:13.171432 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:13.168658    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cc082e2-cebe-49e8-9f30-45f983a493cb" containerName="revoke-keys"
Jul 17 10:18:13.171432 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:13.168668    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cc082e2-cebe-49e8-9f30-45f983a493cb" containerName="revoke-keys"
Jul 17 10:18:13.171432 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:13.168831    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cc082e2-cebe-49e8-9f30-45f983a493cb" containerName="revoke-keys"
Jul 17 10:18:13.171432 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:13.168848    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cc082e2-cebe-49e8-9f30-45f983a493cb" containerName="revoke-keys"
Jul 17 10:18:13.172653 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:13.172629    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vnkn9"
Jul 17 10:18:13.182865 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:13.182830    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vnkn9"]
Jul 17 10:18:13.323051 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:13.323014    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bxrl\" (UniqueName: \"kubernetes.io/projected/818ceebb-f956-4c40-83af-c29cc0661a3c-kube-api-access-6bxrl\") pod \"kuadrant-operator-catalog-vnkn9\" (UID: \"818ceebb-f956-4c40-83af-c29cc0661a3c\") " pod="kuadrant-system/kuadrant-operator-catalog-vnkn9"
Jul 17 10:18:13.424417 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:13.424320    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bxrl\" (UniqueName: \"kubernetes.io/projected/818ceebb-f956-4c40-83af-c29cc0661a3c-kube-api-access-6bxrl\") pod \"kuadrant-operator-catalog-vnkn9\" (UID: \"818ceebb-f956-4c40-83af-c29cc0661a3c\") " pod="kuadrant-system/kuadrant-operator-catalog-vnkn9"
Jul 17 10:18:13.433928 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:13.433889    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bxrl\" (UniqueName: \"kubernetes.io/projected/818ceebb-f956-4c40-83af-c29cc0661a3c-kube-api-access-6bxrl\") pod \"kuadrant-operator-catalog-vnkn9\" (UID: \"818ceebb-f956-4c40-83af-c29cc0661a3c\") " pod="kuadrant-system/kuadrant-operator-catalog-vnkn9"
Jul 17 10:18:13.484990 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:13.484947    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vnkn9"
Jul 17 10:18:13.548548 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:13.548513    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-g27p9"
Jul 17 10:18:13.548734 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:13.548544    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-g27p9" event={"ID":"4cc082e2-cebe-49e8-9f30-45f983a493cb","Type":"ContainerDied","Data":"fcbd69e51cdee9100ff077203e205fd3efc32d8346374238c7108cfec7d4ae72"}
Jul 17 10:18:13.548734 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:13.548647    2574 scope.go:117] "RemoveContainer" containerID="65d5582c64dc812399a4d7b6c2e4163491be03a4bab711e596d48efc535e2f61"
Jul 17 10:18:13.571848 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:13.571813    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-g27p9"]
Jul 17 10:18:13.579288 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:13.579255    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-g27p9"]
Jul 17 10:18:13.644416 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:13.644387    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vnkn9"]
Jul 17 10:18:13.646193 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:18:13.646162    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod818ceebb_f956_4c40_83af_c29cc0661a3c.slice/crio-e7e7060a1187141ad31db084269d73995b91267861f2b09d328aaa280b879b4e WatchSource:0}: Error finding container e7e7060a1187141ad31db084269d73995b91267861f2b09d328aaa280b879b4e: Status 404 returned error can't find the container with id e7e7060a1187141ad31db084269d73995b91267861f2b09d328aaa280b879b4e
Jul 17 10:18:13.647589 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:13.647572    2574 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 17 10:18:14.556537 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:14.556495    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vnkn9" event={"ID":"818ceebb-f956-4c40-83af-c29cc0661a3c","Type":"ContainerStarted","Data":"d2d6abbed0600444d7f7948efba4cdb2d224512bbb31bf9e33abd498da1f06a7"}
Jul 17 10:18:14.557015 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:14.556570    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vnkn9" event={"ID":"818ceebb-f956-4c40-83af-c29cc0661a3c","Type":"ContainerStarted","Data":"e7e7060a1187141ad31db084269d73995b91267861f2b09d328aaa280b879b4e"}
Jul 17 10:18:14.572676 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:14.572620    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-vnkn9" podStartSLOduration=1.175944465 podStartE2EDuration="1.572604735s" totalImagesPullingTime="396.66027ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:18:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:18:14.572127568 +0000 UTC m=+3092.154765944" watchObservedRunningTime="2026-07-17 10:18:14.572604735 +0000 UTC m=+3092.155243111"
Jul 17 10:18:14.897972 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:14.897878    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cc082e2-cebe-49e8-9f30-45f983a493cb" path="/var/lib/kubelet/pods/4cc082e2-cebe-49e8-9f30-45f983a493cb/volumes"
Jul 17 10:18:15.891598 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:15.891564    2574 scope.go:117] "RemoveContainer" containerID="e8120f3f6580f84c641eeb5e203de9933ba023896c5d1fba15c94925a8bb1cea"
Jul 17 10:18:16.571967 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:16.571931    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn" event={"ID":"498055ea-fdc2-4a1e-8bba-6f92d8675e39","Type":"ContainerStarted","Data":"5272d787b9af7606f287269d5307c7a531797103e52b078947a94ce1bbdc4d31"}
Jul 17 10:18:16.588405 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:16.588333    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn" podStartSLOduration=18.588315998 podStartE2EDuration="18.588315998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:17:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:18:16.587168744 +0000 UTC m=+3094.169807122" watchObservedRunningTime="2026-07-17 10:18:16.588315998 +0000 UTC m=+3094.170954384"
Jul 17 10:18:16.601676 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:16.601636    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj"]
Jul 17 10:18:16.602519 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:16.602496    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cc082e2-cebe-49e8-9f30-45f983a493cb" containerName="revoke-keys"
Jul 17 10:18:16.609714 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:16.609683    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj"
Jul 17 10:18:16.612442 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:16.612406    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj"]
Jul 17 10:18:16.763497 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:16.763376    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj\" (UID: \"cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj"
Jul 17 10:18:16.864230 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:16.864186    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj\" (UID: \"cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj"
Jul 17 10:18:16.864909 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:16.864886    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj\" (UID: \"cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj"
Jul 17 10:18:16.973962 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:16.973912    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj"
Jul 17 10:18:17.132721 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:17.132684    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj"]
Jul 17 10:18:17.134634 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:18:17.134598    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcea6bbf3_6a3e_4ddd_9cbd_374044ff54c1.slice/crio-1ea1d7e78cf653304a642475314e389a4fab2948da786872bf6740eb6e827f20 WatchSource:0}: Error finding container 1ea1d7e78cf653304a642475314e389a4fab2948da786872bf6740eb6e827f20: Status 404 returned error can't find the container with id 1ea1d7e78cf653304a642475314e389a4fab2948da786872bf6740eb6e827f20
Jul 17 10:18:17.580136 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:17.580019    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj" event={"ID":"cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1","Type":"ContainerStarted","Data":"aff4847fe900ef81592c831b9f2efa2c2ccac176b4a46fee8a0c9d7370c50006"}
Jul 17 10:18:17.580136 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:17.580080    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj" event={"ID":"cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1","Type":"ContainerStarted","Data":"1ea1d7e78cf653304a642475314e389a4fab2948da786872bf6740eb6e827f20"}
Jul 17 10:18:17.582153 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:17.582122    2574 generic.go:356] "Generic (PLEG): container finished" podID="498055ea-fdc2-4a1e-8bba-6f92d8675e39" containerID="5272d787b9af7606f287269d5307c7a531797103e52b078947a94ce1bbdc4d31" exitCode=6
Jul 17 10:18:17.582342 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:17.582197    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn" event={"ID":"498055ea-fdc2-4a1e-8bba-6f92d8675e39","Type":"ContainerDied","Data":"5272d787b9af7606f287269d5307c7a531797103e52b078947a94ce1bbdc4d31"}
Jul 17 10:18:17.582342 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:17.582274    2574 scope.go:117] "RemoveContainer" containerID="e8120f3f6580f84c641eeb5e203de9933ba023896c5d1fba15c94925a8bb1cea"
Jul 17 10:18:17.582549 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:17.582533    2574 scope.go:117] "RemoveContainer" containerID="5272d787b9af7606f287269d5307c7a531797103e52b078947a94ce1bbdc4d31"
Jul 17 10:18:17.582815 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:18:17.582794    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn_odh-ai-gateway-infra(498055ea-fdc2-4a1e-8bba-6f92d8675e39)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn" podUID="498055ea-fdc2-4a1e-8bba-6f92d8675e39"
Jul 17 10:18:17.597499 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:17.597453    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn"]
Jul 17 10:18:17.599922 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:17.599866    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj" podStartSLOduration=1.599847665 podStartE2EDuration="1.599847665s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:18:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:18:17.597156754 +0000 UTC m=+3095.179795143" watchObservedRunningTime="2026-07-17 10:18:17.599847665 +0000 UTC m=+3095.182486038"
Jul 17 10:18:18.589575 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:18.589541    2574 generic.go:356] "Generic (PLEG): container finished" podID="cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1" containerID="aff4847fe900ef81592c831b9f2efa2c2ccac176b4a46fee8a0c9d7370c50006" exitCode=6
Jul 17 10:18:18.590077 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:18.589627    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj" event={"ID":"cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1","Type":"ContainerDied","Data":"aff4847fe900ef81592c831b9f2efa2c2ccac176b4a46fee8a0c9d7370c50006"}
Jul 17 10:18:18.590077 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:18.589836    2574 scope.go:117] "RemoveContainer" containerID="aff4847fe900ef81592c831b9f2efa2c2ccac176b4a46fee8a0c9d7370c50006"
Jul 17 10:18:18.828328 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:18.828299    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn"
Jul 17 10:18:18.884270 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:18.884132    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/498055ea-fdc2-4a1e-8bba-6f92d8675e39-maas-api-service-ca\") pod \"498055ea-fdc2-4a1e-8bba-6f92d8675e39\" (UID: \"498055ea-fdc2-4a1e-8bba-6f92d8675e39\") "
Jul 17 10:18:18.884580 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:18.884555    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/498055ea-fdc2-4a1e-8bba-6f92d8675e39-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "498055ea-fdc2-4a1e-8bba-6f92d8675e39" (UID: "498055ea-fdc2-4a1e-8bba-6f92d8675e39"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:18:18.985324 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:18.985279    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/498055ea-fdc2-4a1e-8bba-6f92d8675e39-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:18:19.599973 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:19.599938    2574 generic.go:356] "Generic (PLEG): container finished" podID="cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1" containerID="08430a01795acd7e9d92d2deec4160ccfa3540ba21295f6556aac52facefbfb9" exitCode=6
Jul 17 10:18:19.600478 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:19.600033    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj" event={"ID":"cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1","Type":"ContainerDied","Data":"08430a01795acd7e9d92d2deec4160ccfa3540ba21295f6556aac52facefbfb9"}
Jul 17 10:18:19.600478 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:19.600117    2574 scope.go:117] "RemoveContainer" containerID="aff4847fe900ef81592c831b9f2efa2c2ccac176b4a46fee8a0c9d7370c50006"
Jul 17 10:18:19.600478 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:19.600293    2574 scope.go:117] "RemoveContainer" containerID="08430a01795acd7e9d92d2deec4160ccfa3540ba21295f6556aac52facefbfb9"
Jul 17 10:18:19.600659 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:18:19.600575    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj_odh-ai-gateway-infra(cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj" podUID="cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1"
Jul 17 10:18:19.601851 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:19.601832    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn"
Jul 17 10:18:19.601977 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:19.601866    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn" event={"ID":"498055ea-fdc2-4a1e-8bba-6f92d8675e39","Type":"ContainerDied","Data":"78ccbcee00d0980696fba62a4abc8721ba1e033198f2586fbceabedc3f2085ae"}
Jul 17 10:18:19.613177 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:19.613151    2574 scope.go:117] "RemoveContainer" containerID="5272d787b9af7606f287269d5307c7a531797103e52b078947a94ce1bbdc4d31"
Jul 17 10:18:19.631625 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:19.631593    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn"]
Jul 17 10:18:19.636175 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:19.636143    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-nc7qn"]
Jul 17 10:18:20.611544 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:20.611517    2574 scope.go:117] "RemoveContainer" containerID="08430a01795acd7e9d92d2deec4160ccfa3540ba21295f6556aac52facefbfb9"
Jul 17 10:18:20.611999 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:18:20.611768    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj_odh-ai-gateway-infra(cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj" podUID="cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1"
Jul 17 10:18:20.898812 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:20.898724    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="498055ea-fdc2-4a1e-8bba-6f92d8675e39" path="/var/lib/kubelet/pods/498055ea-fdc2-4a1e-8bba-6f92d8675e39/volumes"
Jul 17 10:18:23.485344 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:23.485277    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-vnkn9"
Jul 17 10:18:23.485903 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:23.485381    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-vnkn9"
Jul 17 10:18:23.510054 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:23.510027    2574 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-vnkn9"
Jul 17 10:18:23.655188 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:23.655151    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-vnkn9"
Jul 17 10:18:23.738985 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:23.738878    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vnkn9"]
Jul 17 10:18:25.643075 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:25.643031    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-vnkn9" podUID="818ceebb-f956-4c40-83af-c29cc0661a3c" containerName="registry-server" containerID="cri-o://d2d6abbed0600444d7f7948efba4cdb2d224512bbb31bf9e33abd498da1f06a7" gracePeriod=2
Jul 17 10:18:25.919627 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:25.919595    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vnkn9"
Jul 17 10:18:25.946280 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:25.946212    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6bxrl\" (UniqueName: \"kubernetes.io/projected/818ceebb-f956-4c40-83af-c29cc0661a3c-kube-api-access-6bxrl\") pod \"818ceebb-f956-4c40-83af-c29cc0661a3c\" (UID: \"818ceebb-f956-4c40-83af-c29cc0661a3c\") "
Jul 17 10:18:25.949094 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:25.949053    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/818ceebb-f956-4c40-83af-c29cc0661a3c-kube-api-access-6bxrl" (OuterVolumeSpecName: "kube-api-access-6bxrl") pod "818ceebb-f956-4c40-83af-c29cc0661a3c" (UID: "818ceebb-f956-4c40-83af-c29cc0661a3c"). InnerVolumeSpecName "kube-api-access-6bxrl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:18:26.048073 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:26.048028    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6bxrl\" (UniqueName: \"kubernetes.io/projected/818ceebb-f956-4c40-83af-c29cc0661a3c-kube-api-access-6bxrl\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:18:26.651441 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:26.651405    2574 generic.go:356] "Generic (PLEG): container finished" podID="818ceebb-f956-4c40-83af-c29cc0661a3c" containerID="d2d6abbed0600444d7f7948efba4cdb2d224512bbb31bf9e33abd498da1f06a7" exitCode=0
Jul 17 10:18:26.651901 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:26.651450    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vnkn9"
Jul 17 10:18:26.651901 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:26.651497    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vnkn9" event={"ID":"818ceebb-f956-4c40-83af-c29cc0661a3c","Type":"ContainerDied","Data":"d2d6abbed0600444d7f7948efba4cdb2d224512bbb31bf9e33abd498da1f06a7"}
Jul 17 10:18:26.651901 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:26.651550    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vnkn9" event={"ID":"818ceebb-f956-4c40-83af-c29cc0661a3c","Type":"ContainerDied","Data":"e7e7060a1187141ad31db084269d73995b91267861f2b09d328aaa280b879b4e"}
Jul 17 10:18:26.651901 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:26.651575    2574 scope.go:117] "RemoveContainer" containerID="d2d6abbed0600444d7f7948efba4cdb2d224512bbb31bf9e33abd498da1f06a7"
Jul 17 10:18:26.664091 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:26.664071    2574 scope.go:117] "RemoveContainer" containerID="d2d6abbed0600444d7f7948efba4cdb2d224512bbb31bf9e33abd498da1f06a7"
Jul 17 10:18:26.664472 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:18:26.664449    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2d6abbed0600444d7f7948efba4cdb2d224512bbb31bf9e33abd498da1f06a7\": container with ID starting with d2d6abbed0600444d7f7948efba4cdb2d224512bbb31bf9e33abd498da1f06a7 not found: ID does not exist" containerID="d2d6abbed0600444d7f7948efba4cdb2d224512bbb31bf9e33abd498da1f06a7"
Jul 17 10:18:26.664572 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:26.664487    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2d6abbed0600444d7f7948efba4cdb2d224512bbb31bf9e33abd498da1f06a7"} err="failed to get container status \"d2d6abbed0600444d7f7948efba4cdb2d224512bbb31bf9e33abd498da1f06a7\": rpc error: code = NotFound desc = could not find container \"d2d6abbed0600444d7f7948efba4cdb2d224512bbb31bf9e33abd498da1f06a7\": container with ID starting with d2d6abbed0600444d7f7948efba4cdb2d224512bbb31bf9e33abd498da1f06a7 not found: ID does not exist"
Jul 17 10:18:26.674631 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:26.674591    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vnkn9"]
Jul 17 10:18:26.678032 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:26.678002    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vnkn9"]
Jul 17 10:18:26.898443 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:26.898408    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="818ceebb-f956-4c40-83af-c29cc0661a3c" path="/var/lib/kubelet/pods/818ceebb-f956-4c40-83af-c29cc0661a3c/volumes"
Jul 17 10:18:27.839712 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:27.839662    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2htmf"]
Jul 17 10:18:27.840150 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:27.840136    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="498055ea-fdc2-4a1e-8bba-6f92d8675e39" containerName="revoke-keys"
Jul 17 10:18:27.840196 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:27.840152    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="498055ea-fdc2-4a1e-8bba-6f92d8675e39" containerName="revoke-keys"
Jul 17 10:18:27.840196 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:27.840164    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="818ceebb-f956-4c40-83af-c29cc0661a3c" containerName="registry-server"
Jul 17 10:18:27.840196 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:27.840169    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="818ceebb-f956-4c40-83af-c29cc0661a3c" containerName="registry-server"
Jul 17 10:18:27.840196 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:27.840188    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="498055ea-fdc2-4a1e-8bba-6f92d8675e39" containerName="revoke-keys"
Jul 17 10:18:27.840196 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:27.840194    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="498055ea-fdc2-4a1e-8bba-6f92d8675e39" containerName="revoke-keys"
Jul 17 10:18:27.840372 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:27.840271    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="498055ea-fdc2-4a1e-8bba-6f92d8675e39" containerName="revoke-keys"
Jul 17 10:18:27.840372 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:27.840279    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="498055ea-fdc2-4a1e-8bba-6f92d8675e39" containerName="revoke-keys"
Jul 17 10:18:27.840372 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:27.840289    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="818ceebb-f956-4c40-83af-c29cc0661a3c" containerName="registry-server"
Jul 17 10:18:27.845053 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:27.845028    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2htmf"
Jul 17 10:18:27.850611 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:27.850579    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2htmf"]
Jul 17 10:18:27.867129 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:27.867093    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbf0f444-2069-4d90-81b2-57082496b023-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-2htmf\" (UID: \"fbf0f444-2069-4d90-81b2-57082496b023\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2htmf"
Jul 17 10:18:27.967897 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:27.967847    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbf0f444-2069-4d90-81b2-57082496b023-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-2htmf\" (UID: \"fbf0f444-2069-4d90-81b2-57082496b023\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2htmf"
Jul 17 10:18:27.968585 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:27.968563    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbf0f444-2069-4d90-81b2-57082496b023-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-2htmf\" (UID: \"fbf0f444-2069-4d90-81b2-57082496b023\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2htmf"
Jul 17 10:18:28.159095 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:28.159060    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2htmf"
Jul 17 10:18:28.312271 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:28.312222    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2htmf"]
Jul 17 10:18:28.314477 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:18:28.314446    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbf0f444_2069_4d90_81b2_57082496b023.slice/crio-2f6d82f882fa733e4382bd4247a0b6ba43fc558c3903550c1b8c99bf58cefe34 WatchSource:0}: Error finding container 2f6d82f882fa733e4382bd4247a0b6ba43fc558c3903550c1b8c99bf58cefe34: Status 404 returned error can't find the container with id 2f6d82f882fa733e4382bd4247a0b6ba43fc558c3903550c1b8c99bf58cefe34
Jul 17 10:18:28.665583 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:28.665544    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2htmf" event={"ID":"fbf0f444-2069-4d90-81b2-57082496b023","Type":"ContainerStarted","Data":"47fbce6211ccc608a08aaf011bd788a5d8af058922a415c2c6627a53e8ca72c7"}
Jul 17 10:18:28.665765 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:28.665611    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2htmf" event={"ID":"fbf0f444-2069-4d90-81b2-57082496b023","Type":"ContainerStarted","Data":"2f6d82f882fa733e4382bd4247a0b6ba43fc558c3903550c1b8c99bf58cefe34"}
Jul 17 10:18:28.683143 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:28.683019    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2htmf" podStartSLOduration=1.68299965 podStartE2EDuration="1.68299965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:18:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:18:28.6814533 +0000 UTC m=+3106.264091678" watchObservedRunningTime="2026-07-17 10:18:28.68299965 +0000 UTC m=+3106.265638025"
Jul 17 10:18:29.675231 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:29.675187    2574 generic.go:356] "Generic (PLEG): container finished" podID="fbf0f444-2069-4d90-81b2-57082496b023" containerID="47fbce6211ccc608a08aaf011bd788a5d8af058922a415c2c6627a53e8ca72c7" exitCode=6
Jul 17 10:18:29.675715 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:29.675300    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2htmf" event={"ID":"fbf0f444-2069-4d90-81b2-57082496b023","Type":"ContainerDied","Data":"47fbce6211ccc608a08aaf011bd788a5d8af058922a415c2c6627a53e8ca72c7"}
Jul 17 10:18:29.675715 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:29.675539    2574 scope.go:117] "RemoveContainer" containerID="47fbce6211ccc608a08aaf011bd788a5d8af058922a415c2c6627a53e8ca72c7"
Jul 17 10:18:30.684042 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:30.684005    2574 generic.go:356] "Generic (PLEG): container finished" podID="fbf0f444-2069-4d90-81b2-57082496b023" containerID="94936df728a65f60157bddd2de8f5be79e5e3e9988888987a76cee13de1d1c94" exitCode=6
Jul 17 10:18:30.684665 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:30.684077    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2htmf" event={"ID":"fbf0f444-2069-4d90-81b2-57082496b023","Type":"ContainerDied","Data":"94936df728a65f60157bddd2de8f5be79e5e3e9988888987a76cee13de1d1c94"}
Jul 17 10:18:30.684665 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:30.684136    2574 scope.go:117] "RemoveContainer" containerID="47fbce6211ccc608a08aaf011bd788a5d8af058922a415c2c6627a53e8ca72c7"
Jul 17 10:18:30.684665 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:30.684367    2574 scope.go:117] "RemoveContainer" containerID="94936df728a65f60157bddd2de8f5be79e5e3e9988888987a76cee13de1d1c94"
Jul 17 10:18:30.684843 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:18:30.684672    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-2htmf_odh-ai-gateway-infra(fbf0f444-2069-4d90-81b2-57082496b023)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2htmf" podUID="fbf0f444-2069-4d90-81b2-57082496b023"
Jul 17 10:18:31.692401 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:31.692372    2574 scope.go:117] "RemoveContainer" containerID="94936df728a65f60157bddd2de8f5be79e5e3e9988888987a76cee13de1d1c94"
Jul 17 10:18:31.692814 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:18:31.692581    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-2htmf_odh-ai-gateway-infra(fbf0f444-2069-4d90-81b2-57082496b023)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2htmf" podUID="fbf0f444-2069-4d90-81b2-57082496b023"
Jul 17 10:18:32.895182 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:32.895156    2574 scope.go:117] "RemoveContainer" containerID="08430a01795acd7e9d92d2deec4160ccfa3540ba21295f6556aac52facefbfb9"
Jul 17 10:18:33.706334 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:33.706292    2574 generic.go:356] "Generic (PLEG): container finished" podID="cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1" containerID="e282e99925125320ff2b5dacae27d30ba8202e0f97d6d3998888bfef78c49901" exitCode=6
Jul 17 10:18:33.706507 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:33.706363    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj" event={"ID":"cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1","Type":"ContainerDied","Data":"e282e99925125320ff2b5dacae27d30ba8202e0f97d6d3998888bfef78c49901"}
Jul 17 10:18:33.706507 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:33.706428    2574 scope.go:117] "RemoveContainer" containerID="08430a01795acd7e9d92d2deec4160ccfa3540ba21295f6556aac52facefbfb9"
Jul 17 10:18:33.706825 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:33.706803    2574 scope.go:117] "RemoveContainer" containerID="e282e99925125320ff2b5dacae27d30ba8202e0f97d6d3998888bfef78c49901"
Jul 17 10:18:33.707140 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:18:33.707111    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj_odh-ai-gateway-infra(cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj" podUID="cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1"
Jul 17 10:18:34.732035 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:34.731988    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj"]
Jul 17 10:18:34.907605 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:34.907569    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj"
Jul 17 10:18:34.942551 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:34.942510    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1-maas-api-service-ca\") pod \"cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1\" (UID: \"cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1\") "
Jul 17 10:18:34.942948 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:34.942917    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1" (UID: "cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:18:34.943372 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:34.943345    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:18:35.721704 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:35.721670    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj"
Jul 17 10:18:35.721704 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:35.721700    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj" event={"ID":"cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1","Type":"ContainerDied","Data":"1ea1d7e78cf653304a642475314e389a4fab2948da786872bf6740eb6e827f20"}
Jul 17 10:18:35.721997 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:35.721761    2574 scope.go:117] "RemoveContainer" containerID="e282e99925125320ff2b5dacae27d30ba8202e0f97d6d3998888bfef78c49901"
Jul 17 10:18:35.744556 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:35.744509    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj"]
Jul 17 10:18:35.750761 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:35.750722    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-hnrrj"]
Jul 17 10:18:36.898826 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:36.898793    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1" path="/var/lib/kubelet/pods/cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1/volumes"
Jul 17 10:18:45.891860 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:45.891817    2574 scope.go:117] "RemoveContainer" containerID="94936df728a65f60157bddd2de8f5be79e5e3e9988888987a76cee13de1d1c94"
Jul 17 10:18:46.115090 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:46.115056    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-rpflf"]
Jul 17 10:18:46.115573 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:46.115556    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="498055ea-fdc2-4a1e-8bba-6f92d8675e39" containerName="revoke-keys"
Jul 17 10:18:46.115643 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:46.115576    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="498055ea-fdc2-4a1e-8bba-6f92d8675e39" containerName="revoke-keys"
Jul 17 10:18:46.115643 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:46.115592    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1" containerName="revoke-keys"
Jul 17 10:18:46.115643 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:46.115597    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1" containerName="revoke-keys"
Jul 17 10:18:46.115643 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:46.115614    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1" containerName="revoke-keys"
Jul 17 10:18:46.115643 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:46.115620    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1" containerName="revoke-keys"
Jul 17 10:18:46.115801 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:46.115698    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1" containerName="revoke-keys"
Jul 17 10:18:46.115801 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:46.115707    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1" containerName="revoke-keys"
Jul 17 10:18:46.115801 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:46.115714    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="498055ea-fdc2-4a1e-8bba-6f92d8675e39" containerName="revoke-keys"
Jul 17 10:18:46.115801 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:46.115723    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1" containerName="revoke-keys"
Jul 17 10:18:46.120396 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:46.120376    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-rpflf"
Jul 17 10:18:46.126944 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:46.126910    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-rpflf"]
Jul 17 10:18:46.268396 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:46.268354    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50b141a0-531e-4b99-bdb7-581df8176eb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-rpflf\" (UID: \"50b141a0-531e-4b99-bdb7-581df8176eb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-rpflf"
Jul 17 10:18:46.369433 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:46.369393    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50b141a0-531e-4b99-bdb7-581df8176eb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-rpflf\" (UID: \"50b141a0-531e-4b99-bdb7-581df8176eb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-rpflf"
Jul 17 10:18:46.370092 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:46.370065    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50b141a0-531e-4b99-bdb7-581df8176eb1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-rpflf\" (UID: \"50b141a0-531e-4b99-bdb7-581df8176eb1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-rpflf"
Jul 17 10:18:46.436422 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:46.436378    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-rpflf"
Jul 17 10:18:46.588191 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:46.588164    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-rpflf"]
Jul 17 10:18:46.590027 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:18:46.590000    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50b141a0_531e_4b99_bdb7_581df8176eb1.slice/crio-f49de8af5ed183158245958eca78c118da12f7b2014a5050de81de12e25c795b WatchSource:0}: Error finding container f49de8af5ed183158245958eca78c118da12f7b2014a5050de81de12e25c795b: Status 404 returned error can't find the container with id f49de8af5ed183158245958eca78c118da12f7b2014a5050de81de12e25c795b
Jul 17 10:18:46.788662 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:46.788578    2574 generic.go:356] "Generic (PLEG): container finished" podID="fbf0f444-2069-4d90-81b2-57082496b023" containerID="b5ac96b4ee2c579f279c2f8a95efdbaf86e6c146b4edff5ab2730c5fb432c5f8" exitCode=6
Jul 17 10:18:46.788827 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:46.788657    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2htmf" event={"ID":"fbf0f444-2069-4d90-81b2-57082496b023","Type":"ContainerDied","Data":"b5ac96b4ee2c579f279c2f8a95efdbaf86e6c146b4edff5ab2730c5fb432c5f8"}
Jul 17 10:18:46.788827 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:46.788732    2574 scope.go:117] "RemoveContainer" containerID="94936df728a65f60157bddd2de8f5be79e5e3e9988888987a76cee13de1d1c94"
Jul 17 10:18:46.789079 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:46.789056    2574 scope.go:117] "RemoveContainer" containerID="b5ac96b4ee2c579f279c2f8a95efdbaf86e6c146b4edff5ab2730c5fb432c5f8"
Jul 17 10:18:46.789371 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:18:46.789349    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-2htmf_odh-ai-gateway-infra(fbf0f444-2069-4d90-81b2-57082496b023)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2htmf" podUID="fbf0f444-2069-4d90-81b2-57082496b023"
Jul 17 10:18:46.790172 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:46.790143    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-rpflf" event={"ID":"50b141a0-531e-4b99-bdb7-581df8176eb1","Type":"ContainerStarted","Data":"f49de8af5ed183158245958eca78c118da12f7b2014a5050de81de12e25c795b"}
Jul 17 10:18:47.797946 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:47.797911    2574 generic.go:356] "Generic (PLEG): container finished" podID="50b141a0-531e-4b99-bdb7-581df8176eb1" containerID="37a965d672ebf99f88ef4a07c97fc28885fb74715037f15feb2fbb5acd60c7a4" exitCode=6
Jul 17 10:18:47.798433 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:47.797998    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-rpflf" event={"ID":"50b141a0-531e-4b99-bdb7-581df8176eb1","Type":"ContainerDied","Data":"37a965d672ebf99f88ef4a07c97fc28885fb74715037f15feb2fbb5acd60c7a4"}
Jul 17 10:18:47.798433 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:47.798184    2574 scope.go:117] "RemoveContainer" containerID="37a965d672ebf99f88ef4a07c97fc28885fb74715037f15feb2fbb5acd60c7a4"
Jul 17 10:18:47.819196 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:47.819154    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2htmf"]
Jul 17 10:18:47.965745 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:47.965711    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2htmf"
Jul 17 10:18:48.088501 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:48.088404    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbf0f444-2069-4d90-81b2-57082496b023-maas-api-service-ca\") pod \"fbf0f444-2069-4d90-81b2-57082496b023\" (UID: \"fbf0f444-2069-4d90-81b2-57082496b023\") "
Jul 17 10:18:48.088787 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:48.088763    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fbf0f444-2069-4d90-81b2-57082496b023-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fbf0f444-2069-4d90-81b2-57082496b023" (UID: "fbf0f444-2069-4d90-81b2-57082496b023"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:18:48.190141 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:48.190104    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fbf0f444-2069-4d90-81b2-57082496b023-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:18:48.809840 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:48.809734    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2htmf"
Jul 17 10:18:48.809840 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:48.809821    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2htmf" event={"ID":"fbf0f444-2069-4d90-81b2-57082496b023","Type":"ContainerDied","Data":"2f6d82f882fa733e4382bd4247a0b6ba43fc558c3903550c1b8c99bf58cefe34"}
Jul 17 10:18:48.810425 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:48.809877    2574 scope.go:117] "RemoveContainer" containerID="b5ac96b4ee2c579f279c2f8a95efdbaf86e6c146b4edff5ab2730c5fb432c5f8"
Jul 17 10:18:48.812467 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:48.812433    2574 generic.go:356] "Generic (PLEG): container finished" podID="50b141a0-531e-4b99-bdb7-581df8176eb1" containerID="d1dda4726d5d085a3f3eb559fec38feb6feb01ec1f3a56fc9f5a2a4d0756a9c1" exitCode=6
Jul 17 10:18:48.812621 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:48.812514    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-rpflf" event={"ID":"50b141a0-531e-4b99-bdb7-581df8176eb1","Type":"ContainerDied","Data":"d1dda4726d5d085a3f3eb559fec38feb6feb01ec1f3a56fc9f5a2a4d0756a9c1"}
Jul 17 10:18:48.812809 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:48.812788    2574 scope.go:117] "RemoveContainer" containerID="d1dda4726d5d085a3f3eb559fec38feb6feb01ec1f3a56fc9f5a2a4d0756a9c1"
Jul 17 10:18:48.813090 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:18:48.813063    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-rpflf_odh-ai-gateway-infra(50b141a0-531e-4b99-bdb7-581df8176eb1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-rpflf" podUID="50b141a0-531e-4b99-bdb7-581df8176eb1"
Jul 17 10:18:48.821458 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:48.821436    2574 scope.go:117] "RemoveContainer" containerID="37a965d672ebf99f88ef4a07c97fc28885fb74715037f15feb2fbb5acd60c7a4"
Jul 17 10:18:48.846316 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:48.846279    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2htmf"]
Jul 17 10:18:48.851784 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:48.851753    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-2htmf"]
Jul 17 10:18:48.898524 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:48.898483    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fbf0f444-2069-4d90-81b2-57082496b023" path="/var/lib/kubelet/pods/fbf0f444-2069-4d90-81b2-57082496b023/volumes"
Jul 17 10:18:49.828588 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:49.828556    2574 scope.go:117] "RemoveContainer" containerID="d1dda4726d5d085a3f3eb559fec38feb6feb01ec1f3a56fc9f5a2a4d0756a9c1"
Jul 17 10:18:49.829077 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:18:49.828813    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-rpflf_odh-ai-gateway-infra(50b141a0-531e-4b99-bdb7-581df8176eb1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-rpflf" podUID="50b141a0-531e-4b99-bdb7-581df8176eb1"
Jul 17 10:18:58.785279 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:58.785216    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2"]
Jul 17 10:18:58.785853 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:58.785710    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1" containerName="revoke-keys"
Jul 17 10:18:58.785853 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:58.785724    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="cea6bbf3-6a3e-4ddd-9cbd-374044ff54c1" containerName="revoke-keys"
Jul 17 10:18:58.785853 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:58.785750    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbf0f444-2069-4d90-81b2-57082496b023" containerName="revoke-keys"
Jul 17 10:18:58.785853 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:58.785756    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbf0f444-2069-4d90-81b2-57082496b023" containerName="revoke-keys"
Jul 17 10:18:58.785853 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:58.785762    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbf0f444-2069-4d90-81b2-57082496b023" containerName="revoke-keys"
Jul 17 10:18:58.785853 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:58.785768    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbf0f444-2069-4d90-81b2-57082496b023" containerName="revoke-keys"
Jul 17 10:18:58.785853 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:58.785776    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fbf0f444-2069-4d90-81b2-57082496b023" containerName="revoke-keys"
Jul 17 10:18:58.785853 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:58.785782    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="fbf0f444-2069-4d90-81b2-57082496b023" containerName="revoke-keys"
Jul 17 10:18:58.786118 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:58.785861    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbf0f444-2069-4d90-81b2-57082496b023" containerName="revoke-keys"
Jul 17 10:18:58.786118 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:58.785870    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbf0f444-2069-4d90-81b2-57082496b023" containerName="revoke-keys"
Jul 17 10:18:58.789220 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:58.789201    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2"
Jul 17 10:18:58.796378 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:58.796343    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2"]
Jul 17 10:18:58.891619 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:58.891575    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad15b7d4-e360-418c-896d-709e451f2e03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2\" (UID: \"ad15b7d4-e360-418c-896d-709e451f2e03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2"
Jul 17 10:18:58.992720 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:58.992682    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad15b7d4-e360-418c-896d-709e451f2e03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2\" (UID: \"ad15b7d4-e360-418c-896d-709e451f2e03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2"
Jul 17 10:18:58.993368 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:58.993350    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad15b7d4-e360-418c-896d-709e451f2e03-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2\" (UID: \"ad15b7d4-e360-418c-896d-709e451f2e03\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2"
Jul 17 10:18:59.103818 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:59.103708    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2"
Jul 17 10:18:59.255074 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:59.255045    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2"]
Jul 17 10:18:59.257501 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:18:59.257464    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad15b7d4_e360_418c_896d_709e451f2e03.slice/crio-499fcc06237fc0399b581b5944a276385e683205129423e1f2cbad7f18c7907d WatchSource:0}: Error finding container 499fcc06237fc0399b581b5944a276385e683205129423e1f2cbad7f18c7907d: Status 404 returned error can't find the container with id 499fcc06237fc0399b581b5944a276385e683205129423e1f2cbad7f18c7907d
Jul 17 10:18:59.887136 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:59.887026    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2" event={"ID":"ad15b7d4-e360-418c-896d-709e451f2e03","Type":"ContainerStarted","Data":"d19ef0e41e558d644a3deef628614bcc3fc187fe848dae021209801b527b7153"}
Jul 17 10:18:59.887136 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:59.887098    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2" event={"ID":"ad15b7d4-e360-418c-896d-709e451f2e03","Type":"ContainerStarted","Data":"499fcc06237fc0399b581b5944a276385e683205129423e1f2cbad7f18c7907d"}
Jul 17 10:18:59.906233 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:18:59.906173    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2" podStartSLOduration=1.906152684 podStartE2EDuration="1.906152684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:18:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:18:59.903470512 +0000 UTC m=+3137.486108880" watchObservedRunningTime="2026-07-17 10:18:59.906152684 +0000 UTC m=+3137.488791059"
Jul 17 10:19:00.891717 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:00.891683    2574 scope.go:117] "RemoveContainer" containerID="d1dda4726d5d085a3f3eb559fec38feb6feb01ec1f3a56fc9f5a2a4d0756a9c1"
Jul 17 10:19:00.894667 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:00.894590    2574 generic.go:356] "Generic (PLEG): container finished" podID="ad15b7d4-e360-418c-896d-709e451f2e03" containerID="d19ef0e41e558d644a3deef628614bcc3fc187fe848dae021209801b527b7153" exitCode=6
Jul 17 10:19:00.894884 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:00.894870    2574 scope.go:117] "RemoveContainer" containerID="d19ef0e41e558d644a3deef628614bcc3fc187fe848dae021209801b527b7153"
Jul 17 10:19:00.898827 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:00.898795    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2" event={"ID":"ad15b7d4-e360-418c-896d-709e451f2e03","Type":"ContainerDied","Data":"d19ef0e41e558d644a3deef628614bcc3fc187fe848dae021209801b527b7153"}
Jul 17 10:19:01.903512 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:01.903398    2574 generic.go:356] "Generic (PLEG): container finished" podID="ad15b7d4-e360-418c-896d-709e451f2e03" containerID="5cbc1bbfaa816c0c5b8ac8303724759be8e0f8b7bcb8271e6d026b7c0f42c30f" exitCode=6
Jul 17 10:19:01.903979 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:01.903499    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2" event={"ID":"ad15b7d4-e360-418c-896d-709e451f2e03","Type":"ContainerDied","Data":"5cbc1bbfaa816c0c5b8ac8303724759be8e0f8b7bcb8271e6d026b7c0f42c30f"}
Jul 17 10:19:01.903979 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:01.903576    2574 scope.go:117] "RemoveContainer" containerID="d19ef0e41e558d644a3deef628614bcc3fc187fe848dae021209801b527b7153"
Jul 17 10:19:01.903979 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:01.903894    2574 scope.go:117] "RemoveContainer" containerID="5cbc1bbfaa816c0c5b8ac8303724759be8e0f8b7bcb8271e6d026b7c0f42c30f"
Jul 17 10:19:01.904213 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:19:01.904182    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2_odh-ai-gateway-infra(ad15b7d4-e360-418c-896d-709e451f2e03)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2" podUID="ad15b7d4-e360-418c-896d-709e451f2e03"
Jul 17 10:19:01.906213 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:01.906191    2574 generic.go:356] "Generic (PLEG): container finished" podID="50b141a0-531e-4b99-bdb7-581df8176eb1" containerID="c1e310cdbe3cf97bc2158d5154161b61b0b2ad1d2434925792a8def79f797c20" exitCode=6
Jul 17 10:19:01.906337 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:01.906284    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-rpflf" event={"ID":"50b141a0-531e-4b99-bdb7-581df8176eb1","Type":"ContainerDied","Data":"c1e310cdbe3cf97bc2158d5154161b61b0b2ad1d2434925792a8def79f797c20"}
Jul 17 10:19:01.906667 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:01.906628    2574 scope.go:117] "RemoveContainer" containerID="c1e310cdbe3cf97bc2158d5154161b61b0b2ad1d2434925792a8def79f797c20"
Jul 17 10:19:01.906850 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:19:01.906831    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-rpflf_odh-ai-gateway-infra(50b141a0-531e-4b99-bdb7-581df8176eb1)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-rpflf" podUID="50b141a0-531e-4b99-bdb7-581df8176eb1"
Jul 17 10:19:01.916840 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:01.916813    2574 scope.go:117] "RemoveContainer" containerID="d1dda4726d5d085a3f3eb559fec38feb6feb01ec1f3a56fc9f5a2a4d0756a9c1"
Jul 17 10:19:02.916774 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:02.916745    2574 scope.go:117] "RemoveContainer" containerID="5cbc1bbfaa816c0c5b8ac8303724759be8e0f8b7bcb8271e6d026b7c0f42c30f"
Jul 17 10:19:02.917317 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:19:02.917030    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2_odh-ai-gateway-infra(ad15b7d4-e360-418c-896d-709e451f2e03)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2" podUID="ad15b7d4-e360-418c-896d-709e451f2e03"
Jul 17 10:19:02.947305 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:02.947264    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-rpflf"]
Jul 17 10:19:03.095570 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:03.095543    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-rpflf"
Jul 17 10:19:03.132386 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:03.132352    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50b141a0-531e-4b99-bdb7-581df8176eb1-maas-api-service-ca\") pod \"50b141a0-531e-4b99-bdb7-581df8176eb1\" (UID: \"50b141a0-531e-4b99-bdb7-581df8176eb1\") "
Jul 17 10:19:03.132765 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:03.132740    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50b141a0-531e-4b99-bdb7-581df8176eb1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "50b141a0-531e-4b99-bdb7-581df8176eb1" (UID: "50b141a0-531e-4b99-bdb7-581df8176eb1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:19:03.233921 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:03.233822    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/50b141a0-531e-4b99-bdb7-581df8176eb1-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:19:03.926567 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:03.926530    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-rpflf"
Jul 17 10:19:03.927058 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:03.926537    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-rpflf" event={"ID":"50b141a0-531e-4b99-bdb7-581df8176eb1","Type":"ContainerDied","Data":"f49de8af5ed183158245958eca78c118da12f7b2014a5050de81de12e25c795b"}
Jul 17 10:19:03.927058 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:03.926672    2574 scope.go:117] "RemoveContainer" containerID="c1e310cdbe3cf97bc2158d5154161b61b0b2ad1d2434925792a8def79f797c20"
Jul 17 10:19:03.950937 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:03.950901    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-rpflf"]
Jul 17 10:19:03.958966 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:03.958931    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-rpflf"]
Jul 17 10:19:04.900767 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:04.900719    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50b141a0-531e-4b99-bdb7-581df8176eb1" path="/var/lib/kubelet/pods/50b141a0-531e-4b99-bdb7-581df8176eb1/volumes"
Jul 17 10:19:06.649376 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:06.649332    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qvrxz"]
Jul 17 10:19:06.649858 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:06.649817    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50b141a0-531e-4b99-bdb7-581df8176eb1" containerName="revoke-keys"
Jul 17 10:19:06.649858 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:06.649831    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="50b141a0-531e-4b99-bdb7-581df8176eb1" containerName="revoke-keys"
Jul 17 10:19:06.649858 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:06.649842    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50b141a0-531e-4b99-bdb7-581df8176eb1" containerName="revoke-keys"
Jul 17 10:19:06.649858 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:06.649847    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="50b141a0-531e-4b99-bdb7-581df8176eb1" containerName="revoke-keys"
Jul 17 10:19:06.650012 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:06.649927    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="50b141a0-531e-4b99-bdb7-581df8176eb1" containerName="revoke-keys"
Jul 17 10:19:06.650012 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:06.649940    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="50b141a0-531e-4b99-bdb7-581df8176eb1" containerName="revoke-keys"
Jul 17 10:19:06.650012 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:06.649946    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="50b141a0-531e-4b99-bdb7-581df8176eb1" containerName="revoke-keys"
Jul 17 10:19:06.650012 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:06.649953    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="fbf0f444-2069-4d90-81b2-57082496b023" containerName="revoke-keys"
Jul 17 10:19:06.654451 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:06.654431    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qvrxz"
Jul 17 10:19:06.660226 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:06.660194    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qvrxz"]
Jul 17 10:19:06.767666 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:06.767626    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/112ce25a-9a1e-4d63-b39e-5acee8cdaf1e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-qvrxz\" (UID: \"112ce25a-9a1e-4d63-b39e-5acee8cdaf1e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qvrxz"
Jul 17 10:19:06.868797 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:06.868761    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/112ce25a-9a1e-4d63-b39e-5acee8cdaf1e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-qvrxz\" (UID: \"112ce25a-9a1e-4d63-b39e-5acee8cdaf1e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qvrxz"
Jul 17 10:19:06.869491 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:06.869469    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/112ce25a-9a1e-4d63-b39e-5acee8cdaf1e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-qvrxz\" (UID: \"112ce25a-9a1e-4d63-b39e-5acee8cdaf1e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qvrxz"
Jul 17 10:19:06.968496 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:06.968448    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qvrxz"
Jul 17 10:19:07.122727 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:07.122689    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qvrxz"]
Jul 17 10:19:07.123303 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:19:07.123277    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod112ce25a_9a1e_4d63_b39e_5acee8cdaf1e.slice/crio-b3b1fc7882e953b76a11f9080c6a0f2b29b385f58df58d665335eaa6cf03befb WatchSource:0}: Error finding container b3b1fc7882e953b76a11f9080c6a0f2b29b385f58df58d665335eaa6cf03befb: Status 404 returned error can't find the container with id b3b1fc7882e953b76a11f9080c6a0f2b29b385f58df58d665335eaa6cf03befb
Jul 17 10:19:07.953597 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:07.953553    2574 generic.go:356] "Generic (PLEG): container finished" podID="112ce25a-9a1e-4d63-b39e-5acee8cdaf1e" containerID="262dfd4574e47b3f2616aa329e2792c551909ecbae6e7eb3dc7d32da18acba8b" exitCode=6
Jul 17 10:19:07.954032 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:07.953645    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qvrxz" event={"ID":"112ce25a-9a1e-4d63-b39e-5acee8cdaf1e","Type":"ContainerDied","Data":"262dfd4574e47b3f2616aa329e2792c551909ecbae6e7eb3dc7d32da18acba8b"}
Jul 17 10:19:07.954032 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:07.953699    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qvrxz" event={"ID":"112ce25a-9a1e-4d63-b39e-5acee8cdaf1e","Type":"ContainerStarted","Data":"b3b1fc7882e953b76a11f9080c6a0f2b29b385f58df58d665335eaa6cf03befb"}
Jul 17 10:19:07.954032 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:07.953848    2574 scope.go:117] "RemoveContainer" containerID="262dfd4574e47b3f2616aa329e2792c551909ecbae6e7eb3dc7d32da18acba8b"
Jul 17 10:19:08.961284 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:08.961228    2574 generic.go:356] "Generic (PLEG): container finished" podID="112ce25a-9a1e-4d63-b39e-5acee8cdaf1e" containerID="f1e8f61c8afb46dcf432f43ca90dccfc86fe40a74ca4b52ecdb4a2f56daff9de" exitCode=6
Jul 17 10:19:08.961284 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:08.961274    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qvrxz" event={"ID":"112ce25a-9a1e-4d63-b39e-5acee8cdaf1e","Type":"ContainerDied","Data":"f1e8f61c8afb46dcf432f43ca90dccfc86fe40a74ca4b52ecdb4a2f56daff9de"}
Jul 17 10:19:08.961832 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:08.961341    2574 scope.go:117] "RemoveContainer" containerID="262dfd4574e47b3f2616aa329e2792c551909ecbae6e7eb3dc7d32da18acba8b"
Jul 17 10:19:08.961832 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:08.961614    2574 scope.go:117] "RemoveContainer" containerID="f1e8f61c8afb46dcf432f43ca90dccfc86fe40a74ca4b52ecdb4a2f56daff9de"
Jul 17 10:19:08.961937 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:19:08.961912    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-qvrxz_odh-ai-gateway-infra(112ce25a-9a1e-4d63-b39e-5acee8cdaf1e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qvrxz" podUID="112ce25a-9a1e-4d63-b39e-5acee8cdaf1e"
Jul 17 10:19:09.968908 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:09.968883    2574 scope.go:117] "RemoveContainer" containerID="f1e8f61c8afb46dcf432f43ca90dccfc86fe40a74ca4b52ecdb4a2f56daff9de"
Jul 17 10:19:09.969432 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:19:09.969086    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-qvrxz_odh-ai-gateway-infra(112ce25a-9a1e-4d63-b39e-5acee8cdaf1e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qvrxz" podUID="112ce25a-9a1e-4d63-b39e-5acee8cdaf1e"
Jul 17 10:19:13.891422 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:13.891380    2574 scope.go:117] "RemoveContainer" containerID="5cbc1bbfaa816c0c5b8ac8303724759be8e0f8b7bcb8271e6d026b7c0f42c30f"
Jul 17 10:19:15.001532 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:15.001486    2574 generic.go:356] "Generic (PLEG): container finished" podID="ad15b7d4-e360-418c-896d-709e451f2e03" containerID="ad4f10bb459ed0bf1ed28b32e9b83fa285b580f0e6ad16d5b27274f8040019ff" exitCode=6
Jul 17 10:19:15.002015 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:15.001561    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2" event={"ID":"ad15b7d4-e360-418c-896d-709e451f2e03","Type":"ContainerDied","Data":"ad4f10bb459ed0bf1ed28b32e9b83fa285b580f0e6ad16d5b27274f8040019ff"}
Jul 17 10:19:15.002015 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:15.001623    2574 scope.go:117] "RemoveContainer" containerID="5cbc1bbfaa816c0c5b8ac8303724759be8e0f8b7bcb8271e6d026b7c0f42c30f"
Jul 17 10:19:15.002137 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:15.002055    2574 scope.go:117] "RemoveContainer" containerID="ad4f10bb459ed0bf1ed28b32e9b83fa285b580f0e6ad16d5b27274f8040019ff"
Jul 17 10:19:15.002349 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:19:15.002322    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2_odh-ai-gateway-infra(ad15b7d4-e360-418c-896d-709e451f2e03)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2" podUID="ad15b7d4-e360-418c-896d-709e451f2e03"
Jul 17 10:19:16.030408 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:16.030369    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2"]
Jul 17 10:19:16.183696 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:16.183666    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2"
Jul 17 10:19:16.269013 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:16.268977    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad15b7d4-e360-418c-896d-709e451f2e03-maas-api-service-ca\") pod \"ad15b7d4-e360-418c-896d-709e451f2e03\" (UID: \"ad15b7d4-e360-418c-896d-709e451f2e03\") "
Jul 17 10:19:16.269394 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:16.269372    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad15b7d4-e360-418c-896d-709e451f2e03-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ad15b7d4-e360-418c-896d-709e451f2e03" (UID: "ad15b7d4-e360-418c-896d-709e451f2e03"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:19:16.370028 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:16.369934    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad15b7d4-e360-418c-896d-709e451f2e03-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:19:17.021965 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:17.021929    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2"
Jul 17 10:19:17.021965 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:17.021951    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2" event={"ID":"ad15b7d4-e360-418c-896d-709e451f2e03","Type":"ContainerDied","Data":"499fcc06237fc0399b581b5944a276385e683205129423e1f2cbad7f18c7907d"}
Jul 17 10:19:17.022230 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:17.022023    2574 scope.go:117] "RemoveContainer" containerID="ad4f10bb459ed0bf1ed28b32e9b83fa285b580f0e6ad16d5b27274f8040019ff"
Jul 17 10:19:17.048696 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:17.048651    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2"]
Jul 17 10:19:17.052571 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:17.052539    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-ktpg2"]
Jul 17 10:19:18.898089 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:18.898056    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad15b7d4-e360-418c-896d-709e451f2e03" path="/var/lib/kubelet/pods/ad15b7d4-e360-418c-896d-709e451f2e03/volumes"
Jul 17 10:19:21.891796 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:21.891760    2574 scope.go:117] "RemoveContainer" containerID="f1e8f61c8afb46dcf432f43ca90dccfc86fe40a74ca4b52ecdb4a2f56daff9de"
Jul 17 10:19:23.062449 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:23.062415    2574 generic.go:356] "Generic (PLEG): container finished" podID="112ce25a-9a1e-4d63-b39e-5acee8cdaf1e" containerID="d0b01cb6b430a69bd9926e451bfa53ec3b68acddf6ae3962451ae74f67383cc3" exitCode=6
Jul 17 10:19:23.062922 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:23.062488    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qvrxz" event={"ID":"112ce25a-9a1e-4d63-b39e-5acee8cdaf1e","Type":"ContainerDied","Data":"d0b01cb6b430a69bd9926e451bfa53ec3b68acddf6ae3962451ae74f67383cc3"}
Jul 17 10:19:23.062922 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:23.062557    2574 scope.go:117] "RemoveContainer" containerID="f1e8f61c8afb46dcf432f43ca90dccfc86fe40a74ca4b52ecdb4a2f56daff9de"
Jul 17 10:19:23.063024 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:23.062944    2574 scope.go:117] "RemoveContainer" containerID="d0b01cb6b430a69bd9926e451bfa53ec3b68acddf6ae3962451ae74f67383cc3"
Jul 17 10:19:23.063216 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:19:23.063197    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-qvrxz_odh-ai-gateway-infra(112ce25a-9a1e-4d63-b39e-5acee8cdaf1e)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qvrxz" podUID="112ce25a-9a1e-4d63-b39e-5acee8cdaf1e"
Jul 17 10:19:24.093550 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:24.093508    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qvrxz"]
Jul 17 10:19:24.244271 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:24.244225    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qvrxz"
Jul 17 10:19:24.345829 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:24.345728    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/112ce25a-9a1e-4d63-b39e-5acee8cdaf1e-maas-api-service-ca\") pod \"112ce25a-9a1e-4d63-b39e-5acee8cdaf1e\" (UID: \"112ce25a-9a1e-4d63-b39e-5acee8cdaf1e\") "
Jul 17 10:19:24.346151 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:24.346122    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/112ce25a-9a1e-4d63-b39e-5acee8cdaf1e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "112ce25a-9a1e-4d63-b39e-5acee8cdaf1e" (UID: "112ce25a-9a1e-4d63-b39e-5acee8cdaf1e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:19:24.446894 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:24.446850    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/112ce25a-9a1e-4d63-b39e-5acee8cdaf1e-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:19:25.077914 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:25.077879    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qvrxz"
Jul 17 10:19:25.077914 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:25.077900    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qvrxz" event={"ID":"112ce25a-9a1e-4d63-b39e-5acee8cdaf1e","Type":"ContainerDied","Data":"b3b1fc7882e953b76a11f9080c6a0f2b29b385f58df58d665335eaa6cf03befb"}
Jul 17 10:19:25.078169 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:25.077958    2574 scope.go:117] "RemoveContainer" containerID="d0b01cb6b430a69bd9926e451bfa53ec3b68acddf6ae3962451ae74f67383cc3"
Jul 17 10:19:25.097922 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:25.097881    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qvrxz"]
Jul 17 10:19:25.099960 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:25.099934    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-qvrxz"]
Jul 17 10:19:26.898367 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:26.898331    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="112ce25a-9a1e-4d63-b39e-5acee8cdaf1e" path="/var/lib/kubelet/pods/112ce25a-9a1e-4d63-b39e-5acee8cdaf1e/volumes"
Jul 17 10:19:27.886145 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:27.886101    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr"]
Jul 17 10:19:27.886603 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:27.886588    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad15b7d4-e360-418c-896d-709e451f2e03" containerName="revoke-keys"
Jul 17 10:19:27.886656 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:27.886605    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad15b7d4-e360-418c-896d-709e451f2e03" containerName="revoke-keys"
Jul 17 10:19:27.886656 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:27.886615    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="112ce25a-9a1e-4d63-b39e-5acee8cdaf1e" containerName="revoke-keys"
Jul 17 10:19:27.886656 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:27.886621    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="112ce25a-9a1e-4d63-b39e-5acee8cdaf1e" containerName="revoke-keys"
Jul 17 10:19:27.886656 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:27.886630    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad15b7d4-e360-418c-896d-709e451f2e03" containerName="revoke-keys"
Jul 17 10:19:27.886656 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:27.886637    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad15b7d4-e360-418c-896d-709e451f2e03" containerName="revoke-keys"
Jul 17 10:19:27.886656 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:27.886646    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50b141a0-531e-4b99-bdb7-581df8176eb1" containerName="revoke-keys"
Jul 17 10:19:27.886656 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:27.886651    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="50b141a0-531e-4b99-bdb7-581df8176eb1" containerName="revoke-keys"
Jul 17 10:19:27.886882 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:27.886666    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="112ce25a-9a1e-4d63-b39e-5acee8cdaf1e" containerName="revoke-keys"
Jul 17 10:19:27.886882 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:27.886671    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="112ce25a-9a1e-4d63-b39e-5acee8cdaf1e" containerName="revoke-keys"
Jul 17 10:19:27.886882 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:27.886677    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="112ce25a-9a1e-4d63-b39e-5acee8cdaf1e" containerName="revoke-keys"
Jul 17 10:19:27.886882 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:27.886682    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="112ce25a-9a1e-4d63-b39e-5acee8cdaf1e" containerName="revoke-keys"
Jul 17 10:19:27.886882 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:27.886749    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="112ce25a-9a1e-4d63-b39e-5acee8cdaf1e" containerName="revoke-keys"
Jul 17 10:19:27.886882 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:27.886757    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="112ce25a-9a1e-4d63-b39e-5acee8cdaf1e" containerName="revoke-keys"
Jul 17 10:19:27.886882 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:27.886763    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="112ce25a-9a1e-4d63-b39e-5acee8cdaf1e" containerName="revoke-keys"
Jul 17 10:19:27.886882 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:27.886768    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad15b7d4-e360-418c-896d-709e451f2e03" containerName="revoke-keys"
Jul 17 10:19:27.886882 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:27.886773    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad15b7d4-e360-418c-896d-709e451f2e03" containerName="revoke-keys"
Jul 17 10:19:27.891675 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:27.891647    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr"
Jul 17 10:19:27.895078 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:27.894859    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:19:27.897837 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:27.897791    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr"]
Jul 17 10:19:27.982125 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:27.982069    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d711a8cc-2389-48fc-9608-140f9c5afb43-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr\" (UID: \"d711a8cc-2389-48fc-9608-140f9c5afb43\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr"
Jul 17 10:19:28.082920 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:28.082887    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d711a8cc-2389-48fc-9608-140f9c5afb43-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr\" (UID: \"d711a8cc-2389-48fc-9608-140f9c5afb43\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr"
Jul 17 10:19:28.083770 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:28.083746    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d711a8cc-2389-48fc-9608-140f9c5afb43-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr\" (UID: \"d711a8cc-2389-48fc-9608-140f9c5afb43\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr"
Jul 17 10:19:28.211209 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:28.211170    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr"
Jul 17 10:19:28.368850 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:28.368821    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr"]
Jul 17 10:19:28.371580 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:19:28.371545    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd711a8cc_2389_48fc_9608_140f9c5afb43.slice/crio-9b76cf103a403c488169911be983c4580a0626939759e4eb858948fb301b664a WatchSource:0}: Error finding container 9b76cf103a403c488169911be983c4580a0626939759e4eb858948fb301b664a: Status 404 returned error can't find the container with id 9b76cf103a403c488169911be983c4580a0626939759e4eb858948fb301b664a
Jul 17 10:19:29.104302 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:29.104261    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr" event={"ID":"d711a8cc-2389-48fc-9608-140f9c5afb43","Type":"ContainerStarted","Data":"cd4d97d76f4ee59eb06cf82cad396da19b52cb71d3c830d99445ff20ceb6da47"}
Jul 17 10:19:29.104709 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:29.104338    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr" event={"ID":"d711a8cc-2389-48fc-9608-140f9c5afb43","Type":"ContainerStarted","Data":"9b76cf103a403c488169911be983c4580a0626939759e4eb858948fb301b664a"}
Jul 17 10:19:29.124350 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:29.124293    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr" podStartSLOduration=2.124274333 podStartE2EDuration="2.124274333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:19:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:19:29.12311069 +0000 UTC m=+3166.705749068" watchObservedRunningTime="2026-07-17 10:19:29.124274333 +0000 UTC m=+3166.706912709"
Jul 17 10:19:30.113143 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:30.113106    2574 generic.go:356] "Generic (PLEG): container finished" podID="d711a8cc-2389-48fc-9608-140f9c5afb43" containerID="cd4d97d76f4ee59eb06cf82cad396da19b52cb71d3c830d99445ff20ceb6da47" exitCode=6
Jul 17 10:19:30.113676 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:30.113168    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr" event={"ID":"d711a8cc-2389-48fc-9608-140f9c5afb43","Type":"ContainerDied","Data":"cd4d97d76f4ee59eb06cf82cad396da19b52cb71d3c830d99445ff20ceb6da47"}
Jul 17 10:19:30.113676 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:30.113451    2574 scope.go:117] "RemoveContainer" containerID="cd4d97d76f4ee59eb06cf82cad396da19b52cb71d3c830d99445ff20ceb6da47"
Jul 17 10:19:31.121542 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:31.121500    2574 generic.go:356] "Generic (PLEG): container finished" podID="d711a8cc-2389-48fc-9608-140f9c5afb43" containerID="92fd325d54b824c8e728060d8b6e5de9727622f3b1e37d4ff2201a91a4ee7634" exitCode=6
Jul 17 10:19:31.122024 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:31.121583    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr" event={"ID":"d711a8cc-2389-48fc-9608-140f9c5afb43","Type":"ContainerDied","Data":"92fd325d54b824c8e728060d8b6e5de9727622f3b1e37d4ff2201a91a4ee7634"}
Jul 17 10:19:31.122024 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:31.121645    2574 scope.go:117] "RemoveContainer" containerID="cd4d97d76f4ee59eb06cf82cad396da19b52cb71d3c830d99445ff20ceb6da47"
Jul 17 10:19:31.122024 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:31.121915    2574 scope.go:117] "RemoveContainer" containerID="92fd325d54b824c8e728060d8b6e5de9727622f3b1e37d4ff2201a91a4ee7634"
Jul 17 10:19:31.122192 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:19:31.122128    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr_odh-ai-gateway-infra(d711a8cc-2389-48fc-9608-140f9c5afb43)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr" podUID="d711a8cc-2389-48fc-9608-140f9c5afb43"
Jul 17 10:19:32.128996 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:32.128960    2574 scope.go:117] "RemoveContainer" containerID="92fd325d54b824c8e728060d8b6e5de9727622f3b1e37d4ff2201a91a4ee7634"
Jul 17 10:19:32.129423 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:19:32.129204    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr_odh-ai-gateway-infra(d711a8cc-2389-48fc-9608-140f9c5afb43)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr" podUID="d711a8cc-2389-48fc-9608-140f9c5afb43"
Jul 17 10:19:36.159081 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:36.159032    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnpxn"]
Jul 17 10:19:36.159778 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:36.159754    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad15b7d4-e360-418c-896d-709e451f2e03" containerName="revoke-keys"
Jul 17 10:19:36.159874 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:36.159780    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad15b7d4-e360-418c-896d-709e451f2e03" containerName="revoke-keys"
Jul 17 10:19:36.159930 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:36.159885    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad15b7d4-e360-418c-896d-709e451f2e03" containerName="revoke-keys"
Jul 17 10:19:36.164447 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:36.164421    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnpxn"
Jul 17 10:19:36.170630 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:36.170596    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnpxn"]
Jul 17 10:19:36.267622 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:36.267571    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58a087e0-b4e8-405c-bbd6-e138f2f2ad3a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-gnpxn\" (UID: \"58a087e0-b4e8-405c-bbd6-e138f2f2ad3a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnpxn"
Jul 17 10:19:36.368387 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:36.368336    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58a087e0-b4e8-405c-bbd6-e138f2f2ad3a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-gnpxn\" (UID: \"58a087e0-b4e8-405c-bbd6-e138f2f2ad3a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnpxn"
Jul 17 10:19:36.369031 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:36.369003    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58a087e0-b4e8-405c-bbd6-e138f2f2ad3a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-gnpxn\" (UID: \"58a087e0-b4e8-405c-bbd6-e138f2f2ad3a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnpxn"
Jul 17 10:19:36.478674 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:36.478552    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnpxn"
Jul 17 10:19:36.632110 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:36.632081    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnpxn"]
Jul 17 10:19:36.635119 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:19:36.635080    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58a087e0_b4e8_405c_bbd6_e138f2f2ad3a.slice/crio-a22a20438f186e9a9e06cd9bfd78a09bbdc181ff14511cfddbc10250fc5d1cdf WatchSource:0}: Error finding container a22a20438f186e9a9e06cd9bfd78a09bbdc181ff14511cfddbc10250fc5d1cdf: Status 404 returned error can't find the container with id a22a20438f186e9a9e06cd9bfd78a09bbdc181ff14511cfddbc10250fc5d1cdf
Jul 17 10:19:37.160445 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:37.160413    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnpxn" event={"ID":"58a087e0-b4e8-405c-bbd6-e138f2f2ad3a","Type":"ContainerStarted","Data":"a8dccc42ffd95e7d50f3669816c52932401b4a86f7ac45d2a8e14a62d59ed730"}
Jul 17 10:19:37.160921 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:37.160468    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnpxn" event={"ID":"58a087e0-b4e8-405c-bbd6-e138f2f2ad3a","Type":"ContainerStarted","Data":"a22a20438f186e9a9e06cd9bfd78a09bbdc181ff14511cfddbc10250fc5d1cdf"}
Jul 17 10:19:37.180027 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:37.179970    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnpxn" podStartSLOduration=1.179954037 podStartE2EDuration="1.179954037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:19:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:19:37.178226669 +0000 UTC m=+3174.760865071" watchObservedRunningTime="2026-07-17 10:19:37.179954037 +0000 UTC m=+3174.762592412"
Jul 17 10:19:38.167591 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:38.167554    2574 generic.go:356] "Generic (PLEG): container finished" podID="58a087e0-b4e8-405c-bbd6-e138f2f2ad3a" containerID="a8dccc42ffd95e7d50f3669816c52932401b4a86f7ac45d2a8e14a62d59ed730" exitCode=6
Jul 17 10:19:38.168046 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:38.167647    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnpxn" event={"ID":"58a087e0-b4e8-405c-bbd6-e138f2f2ad3a","Type":"ContainerDied","Data":"a8dccc42ffd95e7d50f3669816c52932401b4a86f7ac45d2a8e14a62d59ed730"}
Jul 17 10:19:38.168046 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:38.167813    2574 scope.go:117] "RemoveContainer" containerID="a8dccc42ffd95e7d50f3669816c52932401b4a86f7ac45d2a8e14a62d59ed730"
Jul 17 10:19:39.175427 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:39.175390    2574 generic.go:356] "Generic (PLEG): container finished" podID="58a087e0-b4e8-405c-bbd6-e138f2f2ad3a" containerID="94a97265231f7b7108ce2f91b4f901254696ee38035ae7ac8302722178e8ee95" exitCode=6
Jul 17 10:19:39.175868 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:39.175478    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnpxn" event={"ID":"58a087e0-b4e8-405c-bbd6-e138f2f2ad3a","Type":"ContainerDied","Data":"94a97265231f7b7108ce2f91b4f901254696ee38035ae7ac8302722178e8ee95"}
Jul 17 10:19:39.175868 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:39.175542    2574 scope.go:117] "RemoveContainer" containerID="a8dccc42ffd95e7d50f3669816c52932401b4a86f7ac45d2a8e14a62d59ed730"
Jul 17 10:19:39.175868 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:39.175841    2574 scope.go:117] "RemoveContainer" containerID="94a97265231f7b7108ce2f91b4f901254696ee38035ae7ac8302722178e8ee95"
Jul 17 10:19:39.176110 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:19:39.176079    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-gnpxn_odh-ai-gateway-infra(58a087e0-b4e8-405c-bbd6-e138f2f2ad3a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnpxn" podUID="58a087e0-b4e8-405c-bbd6-e138f2f2ad3a"
Jul 17 10:19:40.183200 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:40.183169    2574 scope.go:117] "RemoveContainer" containerID="94a97265231f7b7108ce2f91b4f901254696ee38035ae7ac8302722178e8ee95"
Jul 17 10:19:40.183645 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:19:40.183430    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-gnpxn_odh-ai-gateway-infra(58a087e0-b4e8-405c-bbd6-e138f2f2ad3a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnpxn" podUID="58a087e0-b4e8-405c-bbd6-e138f2f2ad3a"
Jul 17 10:19:43.891747 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:43.891713    2574 scope.go:117] "RemoveContainer" containerID="92fd325d54b824c8e728060d8b6e5de9727622f3b1e37d4ff2201a91a4ee7634"
Jul 17 10:19:45.215578 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:45.215542    2574 generic.go:356] "Generic (PLEG): container finished" podID="d711a8cc-2389-48fc-9608-140f9c5afb43" containerID="fc0b315a6ae127154b8c9d5d2e29cbac3796d35f6ae9eabd091dcefb7cf36864" exitCode=6
Jul 17 10:19:45.216038 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:45.215596    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr" event={"ID":"d711a8cc-2389-48fc-9608-140f9c5afb43","Type":"ContainerDied","Data":"fc0b315a6ae127154b8c9d5d2e29cbac3796d35f6ae9eabd091dcefb7cf36864"}
Jul 17 10:19:45.216038 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:45.215655    2574 scope.go:117] "RemoveContainer" containerID="92fd325d54b824c8e728060d8b6e5de9727622f3b1e37d4ff2201a91a4ee7634"
Jul 17 10:19:45.216038 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:45.216002    2574 scope.go:117] "RemoveContainer" containerID="fc0b315a6ae127154b8c9d5d2e29cbac3796d35f6ae9eabd091dcefb7cf36864"
Jul 17 10:19:45.216279 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:19:45.216253    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr_odh-ai-gateway-infra(d711a8cc-2389-48fc-9608-140f9c5afb43)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr" podUID="d711a8cc-2389-48fc-9608-140f9c5afb43"
Jul 17 10:19:46.243016 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:46.242972    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr"]
Jul 17 10:19:46.409700 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:46.409671    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr"
Jul 17 10:19:46.465621 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:46.465578    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d711a8cc-2389-48fc-9608-140f9c5afb43-maas-api-service-ca\") pod \"d711a8cc-2389-48fc-9608-140f9c5afb43\" (UID: \"d711a8cc-2389-48fc-9608-140f9c5afb43\") "
Jul 17 10:19:46.465972 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:46.465944    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d711a8cc-2389-48fc-9608-140f9c5afb43-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d711a8cc-2389-48fc-9608-140f9c5afb43" (UID: "d711a8cc-2389-48fc-9608-140f9c5afb43"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:19:46.567183 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:46.567064    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d711a8cc-2389-48fc-9608-140f9c5afb43-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:19:47.231689 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:47.231592    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr"
Jul 17 10:19:47.231689 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:47.231628    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr" event={"ID":"d711a8cc-2389-48fc-9608-140f9c5afb43","Type":"ContainerDied","Data":"9b76cf103a403c488169911be983c4580a0626939759e4eb858948fb301b664a"}
Jul 17 10:19:47.231941 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:47.231701    2574 scope.go:117] "RemoveContainer" containerID="fc0b315a6ae127154b8c9d5d2e29cbac3796d35f6ae9eabd091dcefb7cf36864"
Jul 17 10:19:47.249461 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:47.249422    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr"]
Jul 17 10:19:47.259508 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:47.259471    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-ljfhr"]
Jul 17 10:19:48.818350 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:48.818281    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-8gms5"]
Jul 17 10:19:48.818858 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:48.818841    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d711a8cc-2389-48fc-9608-140f9c5afb43" containerName="revoke-keys"
Jul 17 10:19:48.818927 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:48.818863    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d711a8cc-2389-48fc-9608-140f9c5afb43" containerName="revoke-keys"
Jul 17 10:19:48.818927 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:48.818888    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d711a8cc-2389-48fc-9608-140f9c5afb43" containerName="revoke-keys"
Jul 17 10:19:48.818927 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:48.818894    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d711a8cc-2389-48fc-9608-140f9c5afb43" containerName="revoke-keys"
Jul 17 10:19:48.818927 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:48.818902    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d711a8cc-2389-48fc-9608-140f9c5afb43" containerName="revoke-keys"
Jul 17 10:19:48.818927 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:48.818908    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="d711a8cc-2389-48fc-9608-140f9c5afb43" containerName="revoke-keys"
Jul 17 10:19:48.819090 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:48.818985    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d711a8cc-2389-48fc-9608-140f9c5afb43" containerName="revoke-keys"
Jul 17 10:19:48.819090 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:48.818993    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d711a8cc-2389-48fc-9608-140f9c5afb43" containerName="revoke-keys"
Jul 17 10:19:48.823374 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:48.823350    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-8gms5"
Jul 17 10:19:48.829767 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:48.829732    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-8gms5"]
Jul 17 10:19:48.892093 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:48.892051    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0341389-2914-4db3-9f51-2ffc68637bfa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-8gms5\" (UID: \"e0341389-2914-4db3-9f51-2ffc68637bfa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-8gms5"
Jul 17 10:19:48.899060 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:48.899020    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d711a8cc-2389-48fc-9608-140f9c5afb43" path="/var/lib/kubelet/pods/d711a8cc-2389-48fc-9608-140f9c5afb43/volumes"
Jul 17 10:19:48.993055 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:48.993011    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0341389-2914-4db3-9f51-2ffc68637bfa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-8gms5\" (UID: \"e0341389-2914-4db3-9f51-2ffc68637bfa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-8gms5"
Jul 17 10:19:48.993635 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:48.993614    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0341389-2914-4db3-9f51-2ffc68637bfa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-8gms5\" (UID: \"e0341389-2914-4db3-9f51-2ffc68637bfa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-8gms5"
Jul 17 10:19:49.138028 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:49.137932    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-8gms5"
Jul 17 10:19:49.297010 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:49.296983    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-8gms5"]
Jul 17 10:19:49.299153 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:19:49.299122    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0341389_2914_4db3_9f51_2ffc68637bfa.slice/crio-f9158198ded48a2e83c786dec9d2227b9998e95a6ed6936db23076d682ceb443 WatchSource:0}: Error finding container f9158198ded48a2e83c786dec9d2227b9998e95a6ed6936db23076d682ceb443: Status 404 returned error can't find the container with id f9158198ded48a2e83c786dec9d2227b9998e95a6ed6936db23076d682ceb443
Jul 17 10:19:50.264571 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:50.264528    2574 generic.go:356] "Generic (PLEG): container finished" podID="e0341389-2914-4db3-9f51-2ffc68637bfa" containerID="699df1fd583738b1b3478c6e6276ac5c1f61c3def73814d791c63012e02ee796" exitCode=6
Jul 17 10:19:50.265004 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:50.264605    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-8gms5" event={"ID":"e0341389-2914-4db3-9f51-2ffc68637bfa","Type":"ContainerDied","Data":"699df1fd583738b1b3478c6e6276ac5c1f61c3def73814d791c63012e02ee796"}
Jul 17 10:19:50.265004 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:50.264691    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-8gms5" event={"ID":"e0341389-2914-4db3-9f51-2ffc68637bfa","Type":"ContainerStarted","Data":"f9158198ded48a2e83c786dec9d2227b9998e95a6ed6936db23076d682ceb443"}
Jul 17 10:19:50.265004 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:50.264796    2574 scope.go:117] "RemoveContainer" containerID="699df1fd583738b1b3478c6e6276ac5c1f61c3def73814d791c63012e02ee796"
Jul 17 10:19:51.272672 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:51.272638    2574 generic.go:356] "Generic (PLEG): container finished" podID="e0341389-2914-4db3-9f51-2ffc68637bfa" containerID="9cef7b1b51e16c9d302540ec29727488eac534a4f3d8eaa70f952183668b2c1f" exitCode=6
Jul 17 10:19:51.273160 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:51.272685    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-8gms5" event={"ID":"e0341389-2914-4db3-9f51-2ffc68637bfa","Type":"ContainerDied","Data":"9cef7b1b51e16c9d302540ec29727488eac534a4f3d8eaa70f952183668b2c1f"}
Jul 17 10:19:51.273160 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:51.272745    2574 scope.go:117] "RemoveContainer" containerID="699df1fd583738b1b3478c6e6276ac5c1f61c3def73814d791c63012e02ee796"
Jul 17 10:19:51.273160 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:51.272923    2574 scope.go:117] "RemoveContainer" containerID="9cef7b1b51e16c9d302540ec29727488eac534a4f3d8eaa70f952183668b2c1f"
Jul 17 10:19:51.273333 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:19:51.273221    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-8gms5_odh-ai-gateway-infra(e0341389-2914-4db3-9f51-2ffc68637bfa)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-8gms5" podUID="e0341389-2914-4db3-9f51-2ffc68637bfa"
Jul 17 10:19:51.892141 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:51.892105    2574 scope.go:117] "RemoveContainer" containerID="94a97265231f7b7108ce2f91b4f901254696ee38035ae7ac8302722178e8ee95"
Jul 17 10:19:52.282109 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:52.282076    2574 scope.go:117] "RemoveContainer" containerID="9cef7b1b51e16c9d302540ec29727488eac534a4f3d8eaa70f952183668b2c1f"
Jul 17 10:19:52.282627 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:19:52.282410    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-8gms5_odh-ai-gateway-infra(e0341389-2914-4db3-9f51-2ffc68637bfa)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-8gms5" podUID="e0341389-2914-4db3-9f51-2ffc68637bfa"
Jul 17 10:19:52.283920 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:52.283895    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnpxn" event={"ID":"58a087e0-b4e8-405c-bbd6-e138f2f2ad3a","Type":"ContainerStarted","Data":"64cc3020955dc3f3f8e5b3f7774a1b41759ea15fe5256763cd6d10460163c681"}
Jul 17 10:19:53.291296 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:53.291256    2574 generic.go:356] "Generic (PLEG): container finished" podID="58a087e0-b4e8-405c-bbd6-e138f2f2ad3a" containerID="64cc3020955dc3f3f8e5b3f7774a1b41759ea15fe5256763cd6d10460163c681" exitCode=6
Jul 17 10:19:53.291296 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:53.291283    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnpxn" event={"ID":"58a087e0-b4e8-405c-bbd6-e138f2f2ad3a","Type":"ContainerDied","Data":"64cc3020955dc3f3f8e5b3f7774a1b41759ea15fe5256763cd6d10460163c681"}
Jul 17 10:19:53.291791 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:53.291343    2574 scope.go:117] "RemoveContainer" containerID="94a97265231f7b7108ce2f91b4f901254696ee38035ae7ac8302722178e8ee95"
Jul 17 10:19:53.291791 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:53.291646    2574 scope.go:117] "RemoveContainer" containerID="64cc3020955dc3f3f8e5b3f7774a1b41759ea15fe5256763cd6d10460163c681"
Jul 17 10:19:53.291942 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:19:53.291917    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-gnpxn_odh-ai-gateway-infra(58a087e0-b4e8-405c-bbd6-e138f2f2ad3a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnpxn" podUID="58a087e0-b4e8-405c-bbd6-e138f2f2ad3a"
Jul 17 10:19:53.320985 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:53.320948    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnpxn"]
Jul 17 10:19:54.466129 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:54.466097    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnpxn"
Jul 17 10:19:54.550484 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:54.550443    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58a087e0-b4e8-405c-bbd6-e138f2f2ad3a-maas-api-service-ca\") pod \"58a087e0-b4e8-405c-bbd6-e138f2f2ad3a\" (UID: \"58a087e0-b4e8-405c-bbd6-e138f2f2ad3a\") "
Jul 17 10:19:54.550879 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:54.550856    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58a087e0-b4e8-405c-bbd6-e138f2f2ad3a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "58a087e0-b4e8-405c-bbd6-e138f2f2ad3a" (UID: "58a087e0-b4e8-405c-bbd6-e138f2f2ad3a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:19:54.651661 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:54.651569    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/58a087e0-b4e8-405c-bbd6-e138f2f2ad3a-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:19:55.311669 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:55.311632    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnpxn"
Jul 17 10:19:55.311669 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:55.311657    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnpxn" event={"ID":"58a087e0-b4e8-405c-bbd6-e138f2f2ad3a","Type":"ContainerDied","Data":"a22a20438f186e9a9e06cd9bfd78a09bbdc181ff14511cfddbc10250fc5d1cdf"}
Jul 17 10:19:55.311936 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:55.311716    2574 scope.go:117] "RemoveContainer" containerID="64cc3020955dc3f3f8e5b3f7774a1b41759ea15fe5256763cd6d10460163c681"
Jul 17 10:19:55.330410 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:55.330378    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnpxn"]
Jul 17 10:19:55.332656 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:55.332633    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnpxn"]
Jul 17 10:19:56.898158 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:19:56.898119    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58a087e0-b4e8-405c-bbd6-e138f2f2ad3a" path="/var/lib/kubelet/pods/58a087e0-b4e8-405c-bbd6-e138f2f2ad3a/volumes"
Jul 17 10:20:06.891816 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:06.891781    2574 scope.go:117] "RemoveContainer" containerID="9cef7b1b51e16c9d302540ec29727488eac534a4f3d8eaa70f952183668b2c1f"
Jul 17 10:20:07.384663 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:07.384626    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-8gms5" event={"ID":"e0341389-2914-4db3-9f51-2ffc68637bfa","Type":"ContainerStarted","Data":"a683181f6b80f55b2f2f8ceb464f351a9d7566649ac6f977a913db9379b2d6f1"}
Jul 17 10:20:07.403401 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:07.403323    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-8gms5" podStartSLOduration=19.403304518 podStartE2EDuration="19.403304518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:19:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:20:07.401262072 +0000 UTC m=+3204.983900441" watchObservedRunningTime="2026-07-17 10:20:07.403304518 +0000 UTC m=+3204.985942921"
Jul 17 10:20:08.392203 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:08.392170    2574 generic.go:356] "Generic (PLEG): container finished" podID="e0341389-2914-4db3-9f51-2ffc68637bfa" containerID="a683181f6b80f55b2f2f8ceb464f351a9d7566649ac6f977a913db9379b2d6f1" exitCode=6
Jul 17 10:20:08.392661 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:08.392263    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-8gms5" event={"ID":"e0341389-2914-4db3-9f51-2ffc68637bfa","Type":"ContainerDied","Data":"a683181f6b80f55b2f2f8ceb464f351a9d7566649ac6f977a913db9379b2d6f1"}
Jul 17 10:20:08.392661 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:08.392338    2574 scope.go:117] "RemoveContainer" containerID="9cef7b1b51e16c9d302540ec29727488eac534a4f3d8eaa70f952183668b2c1f"
Jul 17 10:20:08.392757 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:08.392699    2574 scope.go:117] "RemoveContainer" containerID="a683181f6b80f55b2f2f8ceb464f351a9d7566649ac6f977a913db9379b2d6f1"
Jul 17 10:20:08.392975 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:20:08.392956    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-8gms5_odh-ai-gateway-infra(e0341389-2914-4db3-9f51-2ffc68637bfa)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-8gms5" podUID="e0341389-2914-4db3-9f51-2ffc68637bfa"
Jul 17 10:20:08.413064 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:08.413019    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-8gms5"]
Jul 17 10:20:09.554766 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:09.554736    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-8gms5"
Jul 17 10:20:09.702235 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:09.702195    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0341389-2914-4db3-9f51-2ffc68637bfa-maas-api-service-ca\") pod \"e0341389-2914-4db3-9f51-2ffc68637bfa\" (UID: \"e0341389-2914-4db3-9f51-2ffc68637bfa\") "
Jul 17 10:20:09.702710 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:09.702682    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0341389-2914-4db3-9f51-2ffc68637bfa-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e0341389-2914-4db3-9f51-2ffc68637bfa" (UID: "e0341389-2914-4db3-9f51-2ffc68637bfa"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:20:09.803759 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:09.803717    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e0341389-2914-4db3-9f51-2ffc68637bfa-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:20:10.409555 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:10.409522    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-8gms5"
Jul 17 10:20:10.409773 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:10.409556    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-8gms5" event={"ID":"e0341389-2914-4db3-9f51-2ffc68637bfa","Type":"ContainerDied","Data":"f9158198ded48a2e83c786dec9d2227b9998e95a6ed6936db23076d682ceb443"}
Jul 17 10:20:10.409773 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:10.409634    2574 scope.go:117] "RemoveContainer" containerID="a683181f6b80f55b2f2f8ceb464f351a9d7566649ac6f977a913db9379b2d6f1"
Jul 17 10:20:10.436306 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:10.436267    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-8gms5"]
Jul 17 10:20:10.439663 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:10.439635    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-8gms5"]
Jul 17 10:20:10.905907 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:10.905807    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0341389-2914-4db3-9f51-2ffc68637bfa" path="/var/lib/kubelet/pods/e0341389-2914-4db3-9f51-2ffc68637bfa/volumes"
Jul 17 10:20:17.934595 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:17.934558    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-h44dc"]
Jul 17 10:20:17.935018 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:17.935006    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58a087e0-b4e8-405c-bbd6-e138f2f2ad3a" containerName="revoke-keys"
Jul 17 10:20:17.935018 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:17.935019    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="58a087e0-b4e8-405c-bbd6-e138f2f2ad3a" containerName="revoke-keys"
Jul 17 10:20:17.935096 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:17.935037    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58a087e0-b4e8-405c-bbd6-e138f2f2ad3a" containerName="revoke-keys"
Jul 17 10:20:17.935096 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:17.935042    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="58a087e0-b4e8-405c-bbd6-e138f2f2ad3a" containerName="revoke-keys"
Jul 17 10:20:17.935096 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:17.935049    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0341389-2914-4db3-9f51-2ffc68637bfa" containerName="revoke-keys"
Jul 17 10:20:17.935096 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:17.935056    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0341389-2914-4db3-9f51-2ffc68637bfa" containerName="revoke-keys"
Jul 17 10:20:17.935096 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:17.935077    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0341389-2914-4db3-9f51-2ffc68637bfa" containerName="revoke-keys"
Jul 17 10:20:17.935096 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:17.935082    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0341389-2914-4db3-9f51-2ffc68637bfa" containerName="revoke-keys"
Jul 17 10:20:17.935096 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:17.935088    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0341389-2914-4db3-9f51-2ffc68637bfa" containerName="revoke-keys"
Jul 17 10:20:17.935096 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:17.935093    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0341389-2914-4db3-9f51-2ffc68637bfa" containerName="revoke-keys"
Jul 17 10:20:17.935362 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:17.935168    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="58a087e0-b4e8-405c-bbd6-e138f2f2ad3a" containerName="revoke-keys"
Jul 17 10:20:17.935362 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:17.935175    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0341389-2914-4db3-9f51-2ffc68637bfa" containerName="revoke-keys"
Jul 17 10:20:17.935362 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:17.935182    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0341389-2914-4db3-9f51-2ffc68637bfa" containerName="revoke-keys"
Jul 17 10:20:17.935362 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:17.935189    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0341389-2914-4db3-9f51-2ffc68637bfa" containerName="revoke-keys"
Jul 17 10:20:17.935362 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:17.935197    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="58a087e0-b4e8-405c-bbd6-e138f2f2ad3a" containerName="revoke-keys"
Jul 17 10:20:17.935362 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:17.935204    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="d711a8cc-2389-48fc-9608-140f9c5afb43" containerName="revoke-keys"
Jul 17 10:20:17.938614 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:17.938595    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-h44dc"
Jul 17 10:20:17.941742 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:17.941709    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:20:17.947178 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:17.947141    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-h44dc"]
Jul 17 10:20:17.987538 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:17.987501    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/532653ad-03b0-4fef-850b-ed31e0f3cb8f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-h44dc\" (UID: \"532653ad-03b0-4fef-850b-ed31e0f3cb8f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-h44dc"
Jul 17 10:20:18.088136 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:18.088091    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/532653ad-03b0-4fef-850b-ed31e0f3cb8f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-h44dc\" (UID: \"532653ad-03b0-4fef-850b-ed31e0f3cb8f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-h44dc"
Jul 17 10:20:18.088750 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:18.088727    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/532653ad-03b0-4fef-850b-ed31e0f3cb8f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-h44dc\" (UID: \"532653ad-03b0-4fef-850b-ed31e0f3cb8f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-h44dc"
Jul 17 10:20:18.252026 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:18.251927    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-h44dc"
Jul 17 10:20:18.406362 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:18.406330    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-h44dc"]
Jul 17 10:20:18.408924 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:20:18.408895    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod532653ad_03b0_4fef_850b_ed31e0f3cb8f.slice/crio-c79b1c4997ef0f32c24cd3bce06f8bb90ed216a00c375a9cacde6aa4b8c857eb WatchSource:0}: Error finding container c79b1c4997ef0f32c24cd3bce06f8bb90ed216a00c375a9cacde6aa4b8c857eb: Status 404 returned error can't find the container with id c79b1c4997ef0f32c24cd3bce06f8bb90ed216a00c375a9cacde6aa4b8c857eb
Jul 17 10:20:18.461031 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:18.460994    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-h44dc" event={"ID":"532653ad-03b0-4fef-850b-ed31e0f3cb8f","Type":"ContainerStarted","Data":"c79b1c4997ef0f32c24cd3bce06f8bb90ed216a00c375a9cacde6aa4b8c857eb"}
Jul 17 10:20:19.468474 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:19.468435    2574 generic.go:356] "Generic (PLEG): container finished" podID="532653ad-03b0-4fef-850b-ed31e0f3cb8f" containerID="d8f9266e09a8d1fd934e7d1245e583335f688a3e937c0653c5786e6d0f3f370d" exitCode=6
Jul 17 10:20:19.468894 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:19.468531    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-h44dc" event={"ID":"532653ad-03b0-4fef-850b-ed31e0f3cb8f","Type":"ContainerDied","Data":"d8f9266e09a8d1fd934e7d1245e583335f688a3e937c0653c5786e6d0f3f370d"}
Jul 17 10:20:19.468894 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:19.468730    2574 scope.go:117] "RemoveContainer" containerID="d8f9266e09a8d1fd934e7d1245e583335f688a3e937c0653c5786e6d0f3f370d"
Jul 17 10:20:20.484608 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:20.484569    2574 generic.go:356] "Generic (PLEG): container finished" podID="532653ad-03b0-4fef-850b-ed31e0f3cb8f" containerID="bf311b3e29ce71dfb0b5a0cab0974f2fa6d52260424f4c84d7f10d3d9f5080f7" exitCode=6
Jul 17 10:20:20.485093 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:20.484633    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-h44dc" event={"ID":"532653ad-03b0-4fef-850b-ed31e0f3cb8f","Type":"ContainerDied","Data":"bf311b3e29ce71dfb0b5a0cab0974f2fa6d52260424f4c84d7f10d3d9f5080f7"}
Jul 17 10:20:20.485093 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:20.484758    2574 scope.go:117] "RemoveContainer" containerID="d8f9266e09a8d1fd934e7d1245e583335f688a3e937c0653c5786e6d0f3f370d"
Jul 17 10:20:20.485093 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:20.484971    2574 scope.go:117] "RemoveContainer" containerID="bf311b3e29ce71dfb0b5a0cab0974f2fa6d52260424f4c84d7f10d3d9f5080f7"
Jul 17 10:20:20.485317 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:20:20.485251    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-h44dc_odh-ai-gateway-infra(532653ad-03b0-4fef-850b-ed31e0f3cb8f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-h44dc" podUID="532653ad-03b0-4fef-850b-ed31e0f3cb8f"
Jul 17 10:20:21.493133 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:21.493106    2574 scope.go:117] "RemoveContainer" containerID="bf311b3e29ce71dfb0b5a0cab0974f2fa6d52260424f4c84d7f10d3d9f5080f7"
Jul 17 10:20:21.493583 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:20:21.493363    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-h44dc_odh-ai-gateway-infra(532653ad-03b0-4fef-850b-ed31e0f3cb8f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-h44dc" podUID="532653ad-03b0-4fef-850b-ed31e0f3cb8f"
Jul 17 10:20:26.198096 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:26.198051    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fb9lr"]
Jul 17 10:20:26.198632 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:26.198614    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58a087e0-b4e8-405c-bbd6-e138f2f2ad3a" containerName="revoke-keys"
Jul 17 10:20:26.198692 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:26.198635    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="58a087e0-b4e8-405c-bbd6-e138f2f2ad3a" containerName="revoke-keys"
Jul 17 10:20:26.198734 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:26.198700    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="58a087e0-b4e8-405c-bbd6-e138f2f2ad3a" containerName="revoke-keys"
Jul 17 10:20:26.201890 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:26.201858    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fb9lr"
Jul 17 10:20:26.209685 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:26.209650    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fb9lr"]
Jul 17 10:20:26.270513 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:26.270479    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48bb56d0-560f-428a-b816-23dafc51a12c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-fb9lr\" (UID: \"48bb56d0-560f-428a-b816-23dafc51a12c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fb9lr"
Jul 17 10:20:26.371717 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:26.371661    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48bb56d0-560f-428a-b816-23dafc51a12c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-fb9lr\" (UID: \"48bb56d0-560f-428a-b816-23dafc51a12c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fb9lr"
Jul 17 10:20:26.372351 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:26.372328    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48bb56d0-560f-428a-b816-23dafc51a12c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-fb9lr\" (UID: \"48bb56d0-560f-428a-b816-23dafc51a12c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fb9lr"
Jul 17 10:20:26.517091 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:26.516994    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fb9lr"
Jul 17 10:20:26.681191 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:26.681155    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fb9lr"]
Jul 17 10:20:26.683794 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:20:26.683757    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48bb56d0_560f_428a_b816_23dafc51a12c.slice/crio-5d8a8768cfa768ba7f9b6f9680328d203407cc01c726c9fa9f8549468cb6ee12 WatchSource:0}: Error finding container 5d8a8768cfa768ba7f9b6f9680328d203407cc01c726c9fa9f8549468cb6ee12: Status 404 returned error can't find the container with id 5d8a8768cfa768ba7f9b6f9680328d203407cc01c726c9fa9f8549468cb6ee12
Jul 17 10:20:27.532565 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:27.532523    2574 generic.go:356] "Generic (PLEG): container finished" podID="48bb56d0-560f-428a-b816-23dafc51a12c" containerID="988003c063bf3ac8da7d5a4919cf46e70fc767db202d2c912f8e14ca30e0b94b" exitCode=6
Jul 17 10:20:27.532972 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:27.532575    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fb9lr" event={"ID":"48bb56d0-560f-428a-b816-23dafc51a12c","Type":"ContainerDied","Data":"988003c063bf3ac8da7d5a4919cf46e70fc767db202d2c912f8e14ca30e0b94b"}
Jul 17 10:20:27.532972 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:27.532640    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fb9lr" event={"ID":"48bb56d0-560f-428a-b816-23dafc51a12c","Type":"ContainerStarted","Data":"5d8a8768cfa768ba7f9b6f9680328d203407cc01c726c9fa9f8549468cb6ee12"}
Jul 17 10:20:27.532972 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:27.532815    2574 scope.go:117] "RemoveContainer" containerID="988003c063bf3ac8da7d5a4919cf46e70fc767db202d2c912f8e14ca30e0b94b"
Jul 17 10:20:28.539884 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:28.539845    2574 generic.go:356] "Generic (PLEG): container finished" podID="48bb56d0-560f-428a-b816-23dafc51a12c" containerID="7e065d813eae2cf9c65b5ae39789e1695d76d01bb11ecbf315063a602b759a6a" exitCode=6
Jul 17 10:20:28.540333 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:28.539937    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fb9lr" event={"ID":"48bb56d0-560f-428a-b816-23dafc51a12c","Type":"ContainerDied","Data":"7e065d813eae2cf9c65b5ae39789e1695d76d01bb11ecbf315063a602b759a6a"}
Jul 17 10:20:28.540333 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:28.539993    2574 scope.go:117] "RemoveContainer" containerID="988003c063bf3ac8da7d5a4919cf46e70fc767db202d2c912f8e14ca30e0b94b"
Jul 17 10:20:28.540333 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:28.540192    2574 scope.go:117] "RemoveContainer" containerID="7e065d813eae2cf9c65b5ae39789e1695d76d01bb11ecbf315063a602b759a6a"
Jul 17 10:20:28.540577 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:20:28.540503    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-fb9lr_odh-ai-gateway-infra(48bb56d0-560f-428a-b816-23dafc51a12c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fb9lr" podUID="48bb56d0-560f-428a-b816-23dafc51a12c"
Jul 17 10:20:29.548335 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:29.548304    2574 scope.go:117] "RemoveContainer" containerID="7e065d813eae2cf9c65b5ae39789e1695d76d01bb11ecbf315063a602b759a6a"
Jul 17 10:20:29.548814 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:20:29.548611    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-fb9lr_odh-ai-gateway-infra(48bb56d0-560f-428a-b816-23dafc51a12c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fb9lr" podUID="48bb56d0-560f-428a-b816-23dafc51a12c"
Jul 17 10:20:32.899080 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:32.899047    2574 scope.go:117] "RemoveContainer" containerID="bf311b3e29ce71dfb0b5a0cab0974f2fa6d52260424f4c84d7f10d3d9f5080f7"
Jul 17 10:20:33.577888 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:33.577775    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-h44dc" event={"ID":"532653ad-03b0-4fef-850b-ed31e0f3cb8f","Type":"ContainerStarted","Data":"859a7474aed398231ced2b1366a522989af30041791f5044e7165665e8aeb1ef"}
Jul 17 10:20:33.598327 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:33.598270    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-h44dc" podStartSLOduration=16.59823523 podStartE2EDuration="16.59823523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:20:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:20:33.595665668 +0000 UTC m=+3231.178304045" watchObservedRunningTime="2026-07-17 10:20:33.59823523 +0000 UTC m=+3231.180888193"
Jul 17 10:20:34.586984 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:34.586895    2574 generic.go:356] "Generic (PLEG): container finished" podID="532653ad-03b0-4fef-850b-ed31e0f3cb8f" containerID="859a7474aed398231ced2b1366a522989af30041791f5044e7165665e8aeb1ef" exitCode=6
Jul 17 10:20:34.586984 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:34.586964    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-h44dc" event={"ID":"532653ad-03b0-4fef-850b-ed31e0f3cb8f","Type":"ContainerDied","Data":"859a7474aed398231ced2b1366a522989af30041791f5044e7165665e8aeb1ef"}
Jul 17 10:20:34.587530 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:34.587028    2574 scope.go:117] "RemoveContainer" containerID="bf311b3e29ce71dfb0b5a0cab0974f2fa6d52260424f4c84d7f10d3d9f5080f7"
Jul 17 10:20:34.587530 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:34.587393    2574 scope.go:117] "RemoveContainer" containerID="859a7474aed398231ced2b1366a522989af30041791f5044e7165665e8aeb1ef"
Jul 17 10:20:34.587629 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:20:34.587611    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-h44dc_odh-ai-gateway-infra(532653ad-03b0-4fef-850b-ed31e0f3cb8f)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-h44dc" podUID="532653ad-03b0-4fef-850b-ed31e0f3cb8f"
Jul 17 10:20:34.606536 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:34.606494    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-h44dc"]
Jul 17 10:20:35.748145 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:35.748115    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-h44dc"
Jul 17 10:20:35.875145 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:35.875043    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/532653ad-03b0-4fef-850b-ed31e0f3cb8f-maas-api-service-ca\") pod \"532653ad-03b0-4fef-850b-ed31e0f3cb8f\" (UID: \"532653ad-03b0-4fef-850b-ed31e0f3cb8f\") "
Jul 17 10:20:35.875480 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:35.875454    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/532653ad-03b0-4fef-850b-ed31e0f3cb8f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "532653ad-03b0-4fef-850b-ed31e0f3cb8f" (UID: "532653ad-03b0-4fef-850b-ed31e0f3cb8f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:20:35.975958 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:35.975912    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/532653ad-03b0-4fef-850b-ed31e0f3cb8f-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:20:36.602992 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:36.602957    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-h44dc"
Jul 17 10:20:36.603213 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:36.603003    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-h44dc" event={"ID":"532653ad-03b0-4fef-850b-ed31e0f3cb8f","Type":"ContainerDied","Data":"c79b1c4997ef0f32c24cd3bce06f8bb90ed216a00c375a9cacde6aa4b8c857eb"}
Jul 17 10:20:36.603213 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:36.603118    2574 scope.go:117] "RemoveContainer" containerID="859a7474aed398231ced2b1366a522989af30041791f5044e7165665e8aeb1ef"
Jul 17 10:20:36.626484 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:36.626438    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-h44dc"]
Jul 17 10:20:36.632558 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:36.632522    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-h44dc"]
Jul 17 10:20:36.898941 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:36.898855    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="532653ad-03b0-4fef-850b-ed31e0f3cb8f" path="/var/lib/kubelet/pods/532653ad-03b0-4fef-850b-ed31e0f3cb8f/volumes"
Jul 17 10:20:40.891712 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:40.891677    2574 scope.go:117] "RemoveContainer" containerID="7e065d813eae2cf9c65b5ae39789e1695d76d01bb11ecbf315063a602b759a6a"
Jul 17 10:20:41.639342 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:41.639308    2574 generic.go:356] "Generic (PLEG): container finished" podID="48bb56d0-560f-428a-b816-23dafc51a12c" containerID="8053bd56c4887551d72efd081c44812eb46bf6ad64a92027f03d93e4e72a49c5" exitCode=6
Jul 17 10:20:41.639521 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:41.639392    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fb9lr" event={"ID":"48bb56d0-560f-428a-b816-23dafc51a12c","Type":"ContainerDied","Data":"8053bd56c4887551d72efd081c44812eb46bf6ad64a92027f03d93e4e72a49c5"}
Jul 17 10:20:41.639521 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:41.639467    2574 scope.go:117] "RemoveContainer" containerID="7e065d813eae2cf9c65b5ae39789e1695d76d01bb11ecbf315063a602b759a6a"
Jul 17 10:20:41.639794 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:41.639769    2574 scope.go:117] "RemoveContainer" containerID="8053bd56c4887551d72efd081c44812eb46bf6ad64a92027f03d93e4e72a49c5"
Jul 17 10:20:41.640061 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:20:41.640041    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-fb9lr_odh-ai-gateway-infra(48bb56d0-560f-428a-b816-23dafc51a12c)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fb9lr" podUID="48bb56d0-560f-428a-b816-23dafc51a12c"
Jul 17 10:20:42.666498 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:42.666455    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fb9lr"]
Jul 17 10:20:42.824024 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:42.823995    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fb9lr"
Jul 17 10:20:42.951987 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:42.951875    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48bb56d0-560f-428a-b816-23dafc51a12c-maas-api-service-ca\") pod \"48bb56d0-560f-428a-b816-23dafc51a12c\" (UID: \"48bb56d0-560f-428a-b816-23dafc51a12c\") "
Jul 17 10:20:42.952326 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:42.952301    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48bb56d0-560f-428a-b816-23dafc51a12c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "48bb56d0-560f-428a-b816-23dafc51a12c" (UID: "48bb56d0-560f-428a-b816-23dafc51a12c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:20:43.053539 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:43.053509    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/48bb56d0-560f-428a-b816-23dafc51a12c-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:20:43.655170 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:43.655132    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fb9lr"
Jul 17 10:20:43.655170 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:43.655162    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fb9lr" event={"ID":"48bb56d0-560f-428a-b816-23dafc51a12c","Type":"ContainerDied","Data":"5d8a8768cfa768ba7f9b6f9680328d203407cc01c726c9fa9f8549468cb6ee12"}
Jul 17 10:20:43.655439 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:43.655225    2574 scope.go:117] "RemoveContainer" containerID="8053bd56c4887551d72efd081c44812eb46bf6ad64a92027f03d93e4e72a49c5"
Jul 17 10:20:43.679478 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:43.679436    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fb9lr"]
Jul 17 10:20:43.688473 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:43.688440    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-fb9lr"]
Jul 17 10:20:44.898441 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:44.898407    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48bb56d0-560f-428a-b816-23dafc51a12c" path="/var/lib/kubelet/pods/48bb56d0-560f-428a-b816-23dafc51a12c/volumes"
Jul 17 10:20:46.727447 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:46.727409    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk"]
Jul 17 10:20:46.728068 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:46.728046    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48bb56d0-560f-428a-b816-23dafc51a12c" containerName="revoke-keys"
Jul 17 10:20:46.728164 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:46.728071    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="48bb56d0-560f-428a-b816-23dafc51a12c" containerName="revoke-keys"
Jul 17 10:20:46.728164 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:46.728091    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="532653ad-03b0-4fef-850b-ed31e0f3cb8f" containerName="revoke-keys"
Jul 17 10:20:46.728164 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:46.728100    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="532653ad-03b0-4fef-850b-ed31e0f3cb8f" containerName="revoke-keys"
Jul 17 10:20:46.728164 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:46.728117    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="532653ad-03b0-4fef-850b-ed31e0f3cb8f" containerName="revoke-keys"
Jul 17 10:20:46.728164 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:46.728125    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="532653ad-03b0-4fef-850b-ed31e0f3cb8f" containerName="revoke-keys"
Jul 17 10:20:46.728164 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:46.728135    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="532653ad-03b0-4fef-850b-ed31e0f3cb8f" containerName="revoke-keys"
Jul 17 10:20:46.728164 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:46.728143    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="532653ad-03b0-4fef-850b-ed31e0f3cb8f" containerName="revoke-keys"
Jul 17 10:20:46.728164 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:46.728151    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48bb56d0-560f-428a-b816-23dafc51a12c" containerName="revoke-keys"
Jul 17 10:20:46.728164 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:46.728159    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="48bb56d0-560f-428a-b816-23dafc51a12c" containerName="revoke-keys"
Jul 17 10:20:46.728636 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:46.728173    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48bb56d0-560f-428a-b816-23dafc51a12c" containerName="revoke-keys"
Jul 17 10:20:46.728636 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:46.728183    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="48bb56d0-560f-428a-b816-23dafc51a12c" containerName="revoke-keys"
Jul 17 10:20:46.728636 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:46.728332    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="48bb56d0-560f-428a-b816-23dafc51a12c" containerName="revoke-keys"
Jul 17 10:20:46.728636 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:46.728347    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="532653ad-03b0-4fef-850b-ed31e0f3cb8f" containerName="revoke-keys"
Jul 17 10:20:46.728636 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:46.728355    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="48bb56d0-560f-428a-b816-23dafc51a12c" containerName="revoke-keys"
Jul 17 10:20:46.728636 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:46.728365    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="48bb56d0-560f-428a-b816-23dafc51a12c" containerName="revoke-keys"
Jul 17 10:20:46.728636 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:46.728374    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="532653ad-03b0-4fef-850b-ed31e0f3cb8f" containerName="revoke-keys"
Jul 17 10:20:46.728636 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:46.728385    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="532653ad-03b0-4fef-850b-ed31e0f3cb8f" containerName="revoke-keys"
Jul 17 10:20:46.733793 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:46.733770    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk"
Jul 17 10:20:46.736631 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:46.736608    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:20:46.737951 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:46.737926    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk"]
Jul 17 10:20:46.890754 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:46.890694    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0c8e73e-d846-4e17-a99d-7d660416eb53-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk\" (UID: \"c0c8e73e-d846-4e17-a99d-7d660416eb53\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk"
Jul 17 10:20:46.992139 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:46.992039    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0c8e73e-d846-4e17-a99d-7d660416eb53-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk\" (UID: \"c0c8e73e-d846-4e17-a99d-7d660416eb53\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk"
Jul 17 10:20:46.992747 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:46.992724    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0c8e73e-d846-4e17-a99d-7d660416eb53-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk\" (UID: \"c0c8e73e-d846-4e17-a99d-7d660416eb53\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk"
Jul 17 10:20:47.046726 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:47.046696    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk"
Jul 17 10:20:47.202254 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:47.202171    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk"]
Jul 17 10:20:47.203922 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:20:47.203896    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0c8e73e_d846_4e17_a99d_7d660416eb53.slice/crio-6ccb28df49c270d93d9389b8c4168aa3f92ede7e746c59533ab5499cd286cce2 WatchSource:0}: Error finding container 6ccb28df49c270d93d9389b8c4168aa3f92ede7e746c59533ab5499cd286cce2: Status 404 returned error can't find the container with id 6ccb28df49c270d93d9389b8c4168aa3f92ede7e746c59533ab5499cd286cce2
Jul 17 10:20:47.686752 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:47.686708    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk" event={"ID":"c0c8e73e-d846-4e17-a99d-7d660416eb53","Type":"ContainerStarted","Data":"b4e223515e1fc73e8afeb691236513b47faa209723a72e8c9a4f658ea5ce53a6"}
Jul 17 10:20:47.686943 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:47.686805    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk" event={"ID":"c0c8e73e-d846-4e17-a99d-7d660416eb53","Type":"ContainerStarted","Data":"6ccb28df49c270d93d9389b8c4168aa3f92ede7e746c59533ab5499cd286cce2"}
Jul 17 10:20:47.704950 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:47.704867    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk" podStartSLOduration=1.704845715 podStartE2EDuration="1.704845715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:20:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:20:47.702806137 +0000 UTC m=+3245.285444529" watchObservedRunningTime="2026-07-17 10:20:47.704845715 +0000 UTC m=+3245.287484092"
Jul 17 10:20:48.693727 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:48.693692    2574 generic.go:356] "Generic (PLEG): container finished" podID="c0c8e73e-d846-4e17-a99d-7d660416eb53" containerID="b4e223515e1fc73e8afeb691236513b47faa209723a72e8c9a4f658ea5ce53a6" exitCode=6
Jul 17 10:20:48.694160 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:48.693787    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk" event={"ID":"c0c8e73e-d846-4e17-a99d-7d660416eb53","Type":"ContainerDied","Data":"b4e223515e1fc73e8afeb691236513b47faa209723a72e8c9a4f658ea5ce53a6"}
Jul 17 10:20:48.694160 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:48.693986    2574 scope.go:117] "RemoveContainer" containerID="b4e223515e1fc73e8afeb691236513b47faa209723a72e8c9a4f658ea5ce53a6"
Jul 17 10:20:48.866569 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:48.866535    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb"]
Jul 17 10:20:48.871553 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:48.871528    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb"
Jul 17 10:20:48.877682 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:48.877655    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb"]
Jul 17 10:20:49.011764 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:49.011686    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ede0c48-2dad-4cd3-a17b-49ebdf424bed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb\" (UID: \"2ede0c48-2dad-4cd3-a17b-49ebdf424bed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb"
Jul 17 10:20:49.112677 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:49.112647    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ede0c48-2dad-4cd3-a17b-49ebdf424bed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb\" (UID: \"2ede0c48-2dad-4cd3-a17b-49ebdf424bed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb"
Jul 17 10:20:49.113506 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:49.113484    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ede0c48-2dad-4cd3-a17b-49ebdf424bed-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb\" (UID: \"2ede0c48-2dad-4cd3-a17b-49ebdf424bed\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb"
Jul 17 10:20:49.184967 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:49.184937    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb"
Jul 17 10:20:49.334137 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:49.334111    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb"]
Jul 17 10:20:49.337036 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:20:49.337005    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ede0c48_2dad_4cd3_a17b_49ebdf424bed.slice/crio-90e20edcec4569d21d8f335a2212139c2da523a04408854fb71e6f86ea07eab0 WatchSource:0}: Error finding container 90e20edcec4569d21d8f335a2212139c2da523a04408854fb71e6f86ea07eab0: Status 404 returned error can't find the container with id 90e20edcec4569d21d8f335a2212139c2da523a04408854fb71e6f86ea07eab0
Jul 17 10:20:49.702007 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:49.701975    2574 generic.go:356] "Generic (PLEG): container finished" podID="c0c8e73e-d846-4e17-a99d-7d660416eb53" containerID="0f4528e4935d6f5764c58bd90e1e4c662de05d764957b893a8ab3a9b744aade6" exitCode=6
Jul 17 10:20:49.702511 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:49.702049    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk" event={"ID":"c0c8e73e-d846-4e17-a99d-7d660416eb53","Type":"ContainerDied","Data":"0f4528e4935d6f5764c58bd90e1e4c662de05d764957b893a8ab3a9b744aade6"}
Jul 17 10:20:49.702511 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:49.702120    2574 scope.go:117] "RemoveContainer" containerID="b4e223515e1fc73e8afeb691236513b47faa209723a72e8c9a4f658ea5ce53a6"
Jul 17 10:20:49.702511 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:49.702354    2574 scope.go:117] "RemoveContainer" containerID="0f4528e4935d6f5764c58bd90e1e4c662de05d764957b893a8ab3a9b744aade6"
Jul 17 10:20:49.702705 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:20:49.702617    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk_odh-ai-gateway-infra(c0c8e73e-d846-4e17-a99d-7d660416eb53)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk" podUID="c0c8e73e-d846-4e17-a99d-7d660416eb53"
Jul 17 10:20:49.704363 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:49.704339    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb" event={"ID":"2ede0c48-2dad-4cd3-a17b-49ebdf424bed","Type":"ContainerStarted","Data":"3d07129b23d9089adf205c52f89b52c5218ae369da53aaf7ae1d35b8ee329ea4"}
Jul 17 10:20:49.704483 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:49.704391    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb" event={"ID":"2ede0c48-2dad-4cd3-a17b-49ebdf424bed","Type":"ContainerStarted","Data":"90e20edcec4569d21d8f335a2212139c2da523a04408854fb71e6f86ea07eab0"}
Jul 17 10:20:49.736554 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:49.736497    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb" podStartSLOduration=1.7364769610000002 podStartE2EDuration="1.736476961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:20:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:20:49.73563163 +0000 UTC m=+3247.318270007" watchObservedRunningTime="2026-07-17 10:20:49.736476961 +0000 UTC m=+3247.319115339"
Jul 17 10:20:50.712075 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:50.712042    2574 scope.go:117] "RemoveContainer" containerID="0f4528e4935d6f5764c58bd90e1e4c662de05d764957b893a8ab3a9b744aade6"
Jul 17 10:20:50.712593 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:20:50.712320    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk_odh-ai-gateway-infra(c0c8e73e-d846-4e17-a99d-7d660416eb53)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk" podUID="c0c8e73e-d846-4e17-a99d-7d660416eb53"
Jul 17 10:20:50.713435 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:50.713407    2574 generic.go:356] "Generic (PLEG): container finished" podID="2ede0c48-2dad-4cd3-a17b-49ebdf424bed" containerID="3d07129b23d9089adf205c52f89b52c5218ae369da53aaf7ae1d35b8ee329ea4" exitCode=6
Jul 17 10:20:50.713544 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:50.713495    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb" event={"ID":"2ede0c48-2dad-4cd3-a17b-49ebdf424bed","Type":"ContainerDied","Data":"3d07129b23d9089adf205c52f89b52c5218ae369da53aaf7ae1d35b8ee329ea4"}
Jul 17 10:20:50.713634 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:50.713618    2574 scope.go:117] "RemoveContainer" containerID="3d07129b23d9089adf205c52f89b52c5218ae369da53aaf7ae1d35b8ee329ea4"
Jul 17 10:20:51.722362 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:51.722329    2574 generic.go:356] "Generic (PLEG): container finished" podID="2ede0c48-2dad-4cd3-a17b-49ebdf424bed" containerID="412ad88e9c0928096759555b45767c016d3e1dd7af4fd99b52e00140717e62e6" exitCode=6
Jul 17 10:20:51.722803 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:51.722377    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb" event={"ID":"2ede0c48-2dad-4cd3-a17b-49ebdf424bed","Type":"ContainerDied","Data":"412ad88e9c0928096759555b45767c016d3e1dd7af4fd99b52e00140717e62e6"}
Jul 17 10:20:51.722803 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:51.722425    2574 scope.go:117] "RemoveContainer" containerID="3d07129b23d9089adf205c52f89b52c5218ae369da53aaf7ae1d35b8ee329ea4"
Jul 17 10:20:51.723148 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:51.723127    2574 scope.go:117] "RemoveContainer" containerID="412ad88e9c0928096759555b45767c016d3e1dd7af4fd99b52e00140717e62e6"
Jul 17 10:20:51.723715 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:20:51.723676    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb_odh-ai-gateway-infra(2ede0c48-2dad-4cd3-a17b-49ebdf424bed)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb" podUID="2ede0c48-2dad-4cd3-a17b-49ebdf424bed"
Jul 17 10:20:52.730103 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:20:52.730069    2574 scope.go:117] "RemoveContainer" containerID="412ad88e9c0928096759555b45767c016d3e1dd7af4fd99b52e00140717e62e6"
Jul 17 10:20:52.730566 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:20:52.730357    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb_odh-ai-gateway-infra(2ede0c48-2dad-4cd3-a17b-49ebdf424bed)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb" podUID="2ede0c48-2dad-4cd3-a17b-49ebdf424bed"
Jul 17 10:21:01.891662 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:01.891621    2574 scope.go:117] "RemoveContainer" containerID="0f4528e4935d6f5764c58bd90e1e4c662de05d764957b893a8ab3a9b744aade6"
Jul 17 10:21:02.790704 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:02.790610    2574 generic.go:356] "Generic (PLEG): container finished" podID="c0c8e73e-d846-4e17-a99d-7d660416eb53" containerID="c827b8b2d4cf91264f1381f7e1e3e9215e42efe1c051d4f6b143bb64d279fe66" exitCode=6
Jul 17 10:21:02.790704 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:02.790683    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk" event={"ID":"c0c8e73e-d846-4e17-a99d-7d660416eb53","Type":"ContainerDied","Data":"c827b8b2d4cf91264f1381f7e1e3e9215e42efe1c051d4f6b143bb64d279fe66"}
Jul 17 10:21:02.790946 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:02.790747    2574 scope.go:117] "RemoveContainer" containerID="0f4528e4935d6f5764c58bd90e1e4c662de05d764957b893a8ab3a9b744aade6"
Jul 17 10:21:02.791089 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:02.791054    2574 scope.go:117] "RemoveContainer" containerID="c827b8b2d4cf91264f1381f7e1e3e9215e42efe1c051d4f6b143bb64d279fe66"
Jul 17 10:21:02.791389 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:21:02.791365    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk_odh-ai-gateway-infra(c0c8e73e-d846-4e17-a99d-7d660416eb53)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk" podUID="c0c8e73e-d846-4e17-a99d-7d660416eb53"
Jul 17 10:21:03.819874 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:03.819837    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk"]
Jul 17 10:21:03.968465 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:03.968442    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk"
Jul 17 10:21:04.059519 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:04.059486    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0c8e73e-d846-4e17-a99d-7d660416eb53-maas-api-service-ca\") pod \"c0c8e73e-d846-4e17-a99d-7d660416eb53\" (UID: \"c0c8e73e-d846-4e17-a99d-7d660416eb53\") "
Jul 17 10:21:04.059836 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:04.059814    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0c8e73e-d846-4e17-a99d-7d660416eb53-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c0c8e73e-d846-4e17-a99d-7d660416eb53" (UID: "c0c8e73e-d846-4e17-a99d-7d660416eb53"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:21:04.161205 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:04.161173    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0c8e73e-d846-4e17-a99d-7d660416eb53-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:21:04.807543 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:04.807517    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk"
Jul 17 10:21:04.807753 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:04.807515    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk" event={"ID":"c0c8e73e-d846-4e17-a99d-7d660416eb53","Type":"ContainerDied","Data":"6ccb28df49c270d93d9389b8c4168aa3f92ede7e746c59533ab5499cd286cce2"}
Jul 17 10:21:04.807753 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:04.807655    2574 scope.go:117] "RemoveContainer" containerID="c827b8b2d4cf91264f1381f7e1e3e9215e42efe1c051d4f6b143bb64d279fe66"
Jul 17 10:21:04.832534 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:04.832500    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk"]
Jul 17 10:21:04.838253 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:04.838211    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-7nvmk"]
Jul 17 10:21:04.898765 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:04.898734    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0c8e73e-d846-4e17-a99d-7d660416eb53" path="/var/lib/kubelet/pods/c0c8e73e-d846-4e17-a99d-7d660416eb53/volumes"
Jul 17 10:21:07.891852 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:07.891815    2574 scope.go:117] "RemoveContainer" containerID="412ad88e9c0928096759555b45767c016d3e1dd7af4fd99b52e00140717e62e6"
Jul 17 10:21:07.972080 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:07.972040    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-shnt5"]
Jul 17 10:21:07.972519 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:07.972504    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0c8e73e-d846-4e17-a99d-7d660416eb53" containerName="revoke-keys"
Jul 17 10:21:07.972578 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:07.972520    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0c8e73e-d846-4e17-a99d-7d660416eb53" containerName="revoke-keys"
Jul 17 10:21:07.972578 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:07.972544    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0c8e73e-d846-4e17-a99d-7d660416eb53" containerName="revoke-keys"
Jul 17 10:21:07.972578 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:07.972550    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0c8e73e-d846-4e17-a99d-7d660416eb53" containerName="revoke-keys"
Jul 17 10:21:07.972578 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:07.972561    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0c8e73e-d846-4e17-a99d-7d660416eb53" containerName="revoke-keys"
Jul 17 10:21:07.972578 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:07.972568    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0c8e73e-d846-4e17-a99d-7d660416eb53" containerName="revoke-keys"
Jul 17 10:21:07.972770 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:07.972641    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0c8e73e-d846-4e17-a99d-7d660416eb53" containerName="revoke-keys"
Jul 17 10:21:07.972770 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:07.972648    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0c8e73e-d846-4e17-a99d-7d660416eb53" containerName="revoke-keys"
Jul 17 10:21:07.972770 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:07.972655    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0c8e73e-d846-4e17-a99d-7d660416eb53" containerName="revoke-keys"
Jul 17 10:21:07.975624 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:07.975605    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-shnt5"
Jul 17 10:21:07.983852 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:07.983820    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-shnt5"]
Jul 17 10:21:08.096738 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:08.096692    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de23c962-b159-4716-b9ed-c12a8bd92adf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-shnt5\" (UID: \"de23c962-b159-4716-b9ed-c12a8bd92adf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-shnt5"
Jul 17 10:21:08.197618 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:08.197576    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de23c962-b159-4716-b9ed-c12a8bd92adf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-shnt5\" (UID: \"de23c962-b159-4716-b9ed-c12a8bd92adf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-shnt5"
Jul 17 10:21:08.198295 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:08.198273    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de23c962-b159-4716-b9ed-c12a8bd92adf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-shnt5\" (UID: \"de23c962-b159-4716-b9ed-c12a8bd92adf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-shnt5"
Jul 17 10:21:08.289577 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:08.289532    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-shnt5"
Jul 17 10:21:08.443328 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:08.443297    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-shnt5"]
Jul 17 10:21:08.445051 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:21:08.445025    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde23c962_b159_4716_b9ed_c12a8bd92adf.slice/crio-28469756ade951278f2647d0141eb14e1202a14238553ccd8a4016c6a1a772d8 WatchSource:0}: Error finding container 28469756ade951278f2647d0141eb14e1202a14238553ccd8a4016c6a1a772d8: Status 404 returned error can't find the container with id 28469756ade951278f2647d0141eb14e1202a14238553ccd8a4016c6a1a772d8
Jul 17 10:21:08.834577 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:08.834531    2574 generic.go:356] "Generic (PLEG): container finished" podID="2ede0c48-2dad-4cd3-a17b-49ebdf424bed" containerID="c6950cff163600aa240790df557d8ae0c348287782c4560a2f3c920db1fccbc2" exitCode=6
Jul 17 10:21:08.834779 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:08.834608    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb" event={"ID":"2ede0c48-2dad-4cd3-a17b-49ebdf424bed","Type":"ContainerDied","Data":"c6950cff163600aa240790df557d8ae0c348287782c4560a2f3c920db1fccbc2"}
Jul 17 10:21:08.834779 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:08.834682    2574 scope.go:117] "RemoveContainer" containerID="412ad88e9c0928096759555b45767c016d3e1dd7af4fd99b52e00140717e62e6"
Jul 17 10:21:08.835038 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:08.835015    2574 scope.go:117] "RemoveContainer" containerID="c6950cff163600aa240790df557d8ae0c348287782c4560a2f3c920db1fccbc2"
Jul 17 10:21:08.835341 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:21:08.835318    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb_odh-ai-gateway-infra(2ede0c48-2dad-4cd3-a17b-49ebdf424bed)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb" podUID="2ede0c48-2dad-4cd3-a17b-49ebdf424bed"
Jul 17 10:21:08.836138 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:08.836114    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-shnt5" event={"ID":"de23c962-b159-4716-b9ed-c12a8bd92adf","Type":"ContainerStarted","Data":"28469756ade951278f2647d0141eb14e1202a14238553ccd8a4016c6a1a772d8"}
Jul 17 10:21:09.843671 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:09.843574    2574 generic.go:356] "Generic (PLEG): container finished" podID="de23c962-b159-4716-b9ed-c12a8bd92adf" containerID="e78496ebcb19e4f1ab3bad50688a8cdb981be732fa6e517606a86b698561b1b4" exitCode=6
Jul 17 10:21:09.844137 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:09.843665    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-shnt5" event={"ID":"de23c962-b159-4716-b9ed-c12a8bd92adf","Type":"ContainerDied","Data":"e78496ebcb19e4f1ab3bad50688a8cdb981be732fa6e517606a86b698561b1b4"}
Jul 17 10:21:09.844137 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:09.843862    2574 scope.go:117] "RemoveContainer" containerID="e78496ebcb19e4f1ab3bad50688a8cdb981be732fa6e517606a86b698561b1b4"
Jul 17 10:21:09.864351 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:09.864317    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb"]
Jul 17 10:21:10.022518 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:10.022491    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb"
Jul 17 10:21:10.117867 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:10.117756    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ede0c48-2dad-4cd3-a17b-49ebdf424bed-maas-api-service-ca\") pod \"2ede0c48-2dad-4cd3-a17b-49ebdf424bed\" (UID: \"2ede0c48-2dad-4cd3-a17b-49ebdf424bed\") "
Jul 17 10:21:10.118224 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:10.118194    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ede0c48-2dad-4cd3-a17b-49ebdf424bed-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2ede0c48-2dad-4cd3-a17b-49ebdf424bed" (UID: "2ede0c48-2dad-4cd3-a17b-49ebdf424bed"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:21:10.218640 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:10.218606    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ede0c48-2dad-4cd3-a17b-49ebdf424bed-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:21:10.854408 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:10.854373    2574 generic.go:356] "Generic (PLEG): container finished" podID="de23c962-b159-4716-b9ed-c12a8bd92adf" containerID="34fe7c823ca6c857d69f225a9359a6184cc82745feda437922358f8e965fe35d" exitCode=6
Jul 17 10:21:10.854878 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:10.854467    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-shnt5" event={"ID":"de23c962-b159-4716-b9ed-c12a8bd92adf","Type":"ContainerDied","Data":"34fe7c823ca6c857d69f225a9359a6184cc82745feda437922358f8e965fe35d"}
Jul 17 10:21:10.854878 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:10.854533    2574 scope.go:117] "RemoveContainer" containerID="e78496ebcb19e4f1ab3bad50688a8cdb981be732fa6e517606a86b698561b1b4"
Jul 17 10:21:10.854878 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:10.854703    2574 scope.go:117] "RemoveContainer" containerID="34fe7c823ca6c857d69f225a9359a6184cc82745feda437922358f8e965fe35d"
Jul 17 10:21:10.855083 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:21:10.854961    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-shnt5_odh-ai-gateway-infra(de23c962-b159-4716-b9ed-c12a8bd92adf)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-shnt5" podUID="de23c962-b159-4716-b9ed-c12a8bd92adf"
Jul 17 10:21:10.856043 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:10.856021    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb" event={"ID":"2ede0c48-2dad-4cd3-a17b-49ebdf424bed","Type":"ContainerDied","Data":"90e20edcec4569d21d8f335a2212139c2da523a04408854fb71e6f86ea07eab0"}
Jul 17 10:21:10.856135 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:10.856048    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb"
Jul 17 10:21:10.866931 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:10.866904    2574 scope.go:117] "RemoveContainer" containerID="c6950cff163600aa240790df557d8ae0c348287782c4560a2f3c920db1fccbc2"
Jul 17 10:21:10.914604 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:10.914549    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb"]
Jul 17 10:21:10.918031 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:10.917999    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-b2qvb"]
Jul 17 10:21:11.863720 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:11.863689    2574 scope.go:117] "RemoveContainer" containerID="34fe7c823ca6c857d69f225a9359a6184cc82745feda437922358f8e965fe35d"
Jul 17 10:21:11.864198 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:21:11.863937    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-shnt5_odh-ai-gateway-infra(de23c962-b159-4716-b9ed-c12a8bd92adf)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-shnt5" podUID="de23c962-b159-4716-b9ed-c12a8bd92adf"
Jul 17 10:21:12.898124 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:12.898083    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ede0c48-2dad-4cd3-a17b-49ebdf424bed" path="/var/lib/kubelet/pods/2ede0c48-2dad-4cd3-a17b-49ebdf424bed/volumes"
Jul 17 10:21:16.242143 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:16.242101    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-znz2p"]
Jul 17 10:21:16.242640 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:16.242624    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ede0c48-2dad-4cd3-a17b-49ebdf424bed" containerName="revoke-keys"
Jul 17 10:21:16.242640 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:16.242641    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ede0c48-2dad-4cd3-a17b-49ebdf424bed" containerName="revoke-keys"
Jul 17 10:21:16.242730 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:16.242652    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ede0c48-2dad-4cd3-a17b-49ebdf424bed" containerName="revoke-keys"
Jul 17 10:21:16.242730 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:16.242658    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ede0c48-2dad-4cd3-a17b-49ebdf424bed" containerName="revoke-keys"
Jul 17 10:21:16.242730 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:16.242668    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ede0c48-2dad-4cd3-a17b-49ebdf424bed" containerName="revoke-keys"
Jul 17 10:21:16.242730 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:16.242674    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ede0c48-2dad-4cd3-a17b-49ebdf424bed" containerName="revoke-keys"
Jul 17 10:21:16.242878 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:16.242755    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ede0c48-2dad-4cd3-a17b-49ebdf424bed" containerName="revoke-keys"
Jul 17 10:21:16.242878 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:16.242764    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ede0c48-2dad-4cd3-a17b-49ebdf424bed" containerName="revoke-keys"
Jul 17 10:21:16.242878 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:16.242771    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ede0c48-2dad-4cd3-a17b-49ebdf424bed" containerName="revoke-keys"
Jul 17 10:21:16.247339 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:16.247318    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-znz2p"
Jul 17 10:21:16.253273 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:16.253226    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-znz2p"]
Jul 17 10:21:16.382075 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:16.382036    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de4934ee-ae73-4f9f-8747-cdcd88c3dfbe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-znz2p\" (UID: \"de4934ee-ae73-4f9f-8747-cdcd88c3dfbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-znz2p"
Jul 17 10:21:16.483160 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:16.483116    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de4934ee-ae73-4f9f-8747-cdcd88c3dfbe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-znz2p\" (UID: \"de4934ee-ae73-4f9f-8747-cdcd88c3dfbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-znz2p"
Jul 17 10:21:16.483779 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:16.483756    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de4934ee-ae73-4f9f-8747-cdcd88c3dfbe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-znz2p\" (UID: \"de4934ee-ae73-4f9f-8747-cdcd88c3dfbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-znz2p"
Jul 17 10:21:16.561478 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:16.561378    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-znz2p"
Jul 17 10:21:16.921618 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:16.921585    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-znz2p"]
Jul 17 10:21:16.924321 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:21:16.924289    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde4934ee_ae73_4f9f_8747_cdcd88c3dfbe.slice/crio-80bf77024272f48bd1c2cf21b747049adad5260ceb0b8b449c6dba2bddb8e437 WatchSource:0}: Error finding container 80bf77024272f48bd1c2cf21b747049adad5260ceb0b8b449c6dba2bddb8e437: Status 404 returned error can't find the container with id 80bf77024272f48bd1c2cf21b747049adad5260ceb0b8b449c6dba2bddb8e437
Jul 17 10:21:17.901597 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:17.901508    2574 generic.go:356] "Generic (PLEG): container finished" podID="de4934ee-ae73-4f9f-8747-cdcd88c3dfbe" containerID="2ac8496aaa62e7bdfc5d9f1721ce94934821f3210d4fc7619d2ee24a47f64b2c" exitCode=6
Jul 17 10:21:17.902008 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:17.901598    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-znz2p" event={"ID":"de4934ee-ae73-4f9f-8747-cdcd88c3dfbe","Type":"ContainerDied","Data":"2ac8496aaa62e7bdfc5d9f1721ce94934821f3210d4fc7619d2ee24a47f64b2c"}
Jul 17 10:21:17.902008 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:17.901667    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-znz2p" event={"ID":"de4934ee-ae73-4f9f-8747-cdcd88c3dfbe","Type":"ContainerStarted","Data":"80bf77024272f48bd1c2cf21b747049adad5260ceb0b8b449c6dba2bddb8e437"}
Jul 17 10:21:17.902008 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:17.901816    2574 scope.go:117] "RemoveContainer" containerID="2ac8496aaa62e7bdfc5d9f1721ce94934821f3210d4fc7619d2ee24a47f64b2c"
Jul 17 10:21:18.909283 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:18.909214    2574 generic.go:356] "Generic (PLEG): container finished" podID="de4934ee-ae73-4f9f-8747-cdcd88c3dfbe" containerID="98380ae63a310e387478b7da51ed4888d1168d6065e5d2ed22c07612b593b642" exitCode=6
Jul 17 10:21:18.909283 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:18.909275    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-znz2p" event={"ID":"de4934ee-ae73-4f9f-8747-cdcd88c3dfbe","Type":"ContainerDied","Data":"98380ae63a310e387478b7da51ed4888d1168d6065e5d2ed22c07612b593b642"}
Jul 17 10:21:18.909805 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:18.909345    2574 scope.go:117] "RemoveContainer" containerID="2ac8496aaa62e7bdfc5d9f1721ce94934821f3210d4fc7619d2ee24a47f64b2c"
Jul 17 10:21:18.909805 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:18.909540    2574 scope.go:117] "RemoveContainer" containerID="98380ae63a310e387478b7da51ed4888d1168d6065e5d2ed22c07612b593b642"
Jul 17 10:21:18.909909 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:21:18.909810    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-znz2p_odh-ai-gateway-infra(de4934ee-ae73-4f9f-8747-cdcd88c3dfbe)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-znz2p" podUID="de4934ee-ae73-4f9f-8747-cdcd88c3dfbe"
Jul 17 10:21:19.917379 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:19.917350    2574 scope.go:117] "RemoveContainer" containerID="98380ae63a310e387478b7da51ed4888d1168d6065e5d2ed22c07612b593b642"
Jul 17 10:21:19.917782 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:21:19.917604    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-znz2p_odh-ai-gateway-infra(de4934ee-ae73-4f9f-8747-cdcd88c3dfbe)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-znz2p" podUID="de4934ee-ae73-4f9f-8747-cdcd88c3dfbe"
Jul 17 10:21:23.892276 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:23.892212    2574 scope.go:117] "RemoveContainer" containerID="34fe7c823ca6c857d69f225a9359a6184cc82745feda437922358f8e965fe35d"
Jul 17 10:21:24.947953 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:24.947913    2574 generic.go:356] "Generic (PLEG): container finished" podID="de23c962-b159-4716-b9ed-c12a8bd92adf" containerID="ded0b413f4044e2a91c530a5d470253152fea09ec9dee94cacfac98fa59fecd3" exitCode=6
Jul 17 10:21:24.948445 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:24.947986    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-shnt5" event={"ID":"de23c962-b159-4716-b9ed-c12a8bd92adf","Type":"ContainerDied","Data":"ded0b413f4044e2a91c530a5d470253152fea09ec9dee94cacfac98fa59fecd3"}
Jul 17 10:21:24.948445 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:24.948054    2574 scope.go:117] "RemoveContainer" containerID="34fe7c823ca6c857d69f225a9359a6184cc82745feda437922358f8e965fe35d"
Jul 17 10:21:24.948557 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:24.948485    2574 scope.go:117] "RemoveContainer" containerID="ded0b413f4044e2a91c530a5d470253152fea09ec9dee94cacfac98fa59fecd3"
Jul 17 10:21:24.948750 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:21:24.948730    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-shnt5_odh-ai-gateway-infra(de23c962-b159-4716-b9ed-c12a8bd92adf)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-shnt5" podUID="de23c962-b159-4716-b9ed-c12a8bd92adf"
Jul 17 10:21:25.976927 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:25.976888    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-shnt5"]
Jul 17 10:21:26.136095 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:26.136056    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-shnt5"
Jul 17 10:21:26.179147 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:26.179113    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de23c962-b159-4716-b9ed-c12a8bd92adf-maas-api-service-ca\") pod \"de23c962-b159-4716-b9ed-c12a8bd92adf\" (UID: \"de23c962-b159-4716-b9ed-c12a8bd92adf\") "
Jul 17 10:21:26.179551 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:26.179527    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de23c962-b159-4716-b9ed-c12a8bd92adf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "de23c962-b159-4716-b9ed-c12a8bd92adf" (UID: "de23c962-b159-4716-b9ed-c12a8bd92adf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:21:26.280862 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:26.280760    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de23c962-b159-4716-b9ed-c12a8bd92adf-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:21:26.962730 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:26.962696    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-shnt5"
Jul 17 10:21:26.962730 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:26.962725    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-shnt5" event={"ID":"de23c962-b159-4716-b9ed-c12a8bd92adf","Type":"ContainerDied","Data":"28469756ade951278f2647d0141eb14e1202a14238553ccd8a4016c6a1a772d8"}
Jul 17 10:21:26.963011 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:26.962792    2574 scope.go:117] "RemoveContainer" containerID="ded0b413f4044e2a91c530a5d470253152fea09ec9dee94cacfac98fa59fecd3"
Jul 17 10:21:26.980815 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:26.980780    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-shnt5"]
Jul 17 10:21:26.988325 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:26.988291    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-shnt5"]
Jul 17 10:21:28.898513 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:28.898479    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de23c962-b159-4716-b9ed-c12a8bd92adf" path="/var/lib/kubelet/pods/de23c962-b159-4716-b9ed-c12a8bd92adf/volumes"
Jul 17 10:21:30.891837 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:30.891795    2574 scope.go:117] "RemoveContainer" containerID="98380ae63a310e387478b7da51ed4888d1168d6065e5d2ed22c07612b593b642"
Jul 17 10:21:31.997946 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:31.997911    2574 generic.go:356] "Generic (PLEG): container finished" podID="de4934ee-ae73-4f9f-8747-cdcd88c3dfbe" containerID="bf27f2e7e535f82a272effa91d523b261177270f0eed32363d3b7124c7360066" exitCode=6
Jul 17 10:21:31.998493 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:31.997990    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-znz2p" event={"ID":"de4934ee-ae73-4f9f-8747-cdcd88c3dfbe","Type":"ContainerDied","Data":"bf27f2e7e535f82a272effa91d523b261177270f0eed32363d3b7124c7360066"}
Jul 17 10:21:31.998493 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:31.998071    2574 scope.go:117] "RemoveContainer" containerID="98380ae63a310e387478b7da51ed4888d1168d6065e5d2ed22c07612b593b642"
Jul 17 10:21:31.998493 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:31.998446    2574 scope.go:117] "RemoveContainer" containerID="bf27f2e7e535f82a272effa91d523b261177270f0eed32363d3b7124c7360066"
Jul 17 10:21:31.998697 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:21:31.998672    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-znz2p_odh-ai-gateway-infra(de4934ee-ae73-4f9f-8747-cdcd88c3dfbe)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-znz2p" podUID="de4934ee-ae73-4f9f-8747-cdcd88c3dfbe"
Jul 17 10:21:33.034853 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:33.034817    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-znz2p"]
Jul 17 10:21:33.193286 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:33.193249    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-znz2p"
Jul 17 10:21:33.249458 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:33.249416    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de4934ee-ae73-4f9f-8747-cdcd88c3dfbe-maas-api-service-ca\") pod \"de4934ee-ae73-4f9f-8747-cdcd88c3dfbe\" (UID: \"de4934ee-ae73-4f9f-8747-cdcd88c3dfbe\") "
Jul 17 10:21:33.249841 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:33.249809    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de4934ee-ae73-4f9f-8747-cdcd88c3dfbe-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "de4934ee-ae73-4f9f-8747-cdcd88c3dfbe" (UID: "de4934ee-ae73-4f9f-8747-cdcd88c3dfbe"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:21:33.350623 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:33.350522    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/de4934ee-ae73-4f9f-8747-cdcd88c3dfbe-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:21:34.013558 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:34.013519    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-znz2p" event={"ID":"de4934ee-ae73-4f9f-8747-cdcd88c3dfbe","Type":"ContainerDied","Data":"80bf77024272f48bd1c2cf21b747049adad5260ceb0b8b449c6dba2bddb8e437"}
Jul 17 10:21:34.013763 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:34.013586    2574 scope.go:117] "RemoveContainer" containerID="bf27f2e7e535f82a272effa91d523b261177270f0eed32363d3b7124c7360066"
Jul 17 10:21:34.013763 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:34.013538    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-znz2p"
Jul 17 10:21:34.037121 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:34.037084    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-znz2p"]
Jul 17 10:21:34.041205 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:34.041176    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-znz2p"]
Jul 17 10:21:34.898674 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:34.898634    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de4934ee-ae73-4f9f-8747-cdcd88c3dfbe" path="/var/lib/kubelet/pods/de4934ee-ae73-4f9f-8747-cdcd88c3dfbe/volumes"
Jul 17 10:21:43.221648 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:43.221616    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dh564_84db06ab-87e2-43b3-82d0-13f06ba86d06/ovn-acl-logging/0.log"
Jul 17 10:21:43.229864 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:43.229837    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dh564_84db06ab-87e2-43b3-82d0-13f06ba86d06/ovn-acl-logging/0.log"
Jul 17 10:21:48.913873 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:48.913830    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wpj4r"]
Jul 17 10:21:48.914534 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:48.914511    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de4934ee-ae73-4f9f-8747-cdcd88c3dfbe" containerName="revoke-keys"
Jul 17 10:21:48.914630 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:48.914537    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="de4934ee-ae73-4f9f-8747-cdcd88c3dfbe" containerName="revoke-keys"
Jul 17 10:21:48.914630 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:48.914568    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de23c962-b159-4716-b9ed-c12a8bd92adf" containerName="revoke-keys"
Jul 17 10:21:48.914630 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:48.914577    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="de23c962-b159-4716-b9ed-c12a8bd92adf" containerName="revoke-keys"
Jul 17 10:21:48.914630 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:48.914594    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de23c962-b159-4716-b9ed-c12a8bd92adf" containerName="revoke-keys"
Jul 17 10:21:48.914630 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:48.914601    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="de23c962-b159-4716-b9ed-c12a8bd92adf" containerName="revoke-keys"
Jul 17 10:21:48.914630 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:48.914617    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de4934ee-ae73-4f9f-8747-cdcd88c3dfbe" containerName="revoke-keys"
Jul 17 10:21:48.914630 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:48.914625    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="de4934ee-ae73-4f9f-8747-cdcd88c3dfbe" containerName="revoke-keys"
Jul 17 10:21:48.914991 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:48.914635    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de23c962-b159-4716-b9ed-c12a8bd92adf" containerName="revoke-keys"
Jul 17 10:21:48.914991 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:48.914643    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="de23c962-b159-4716-b9ed-c12a8bd92adf" containerName="revoke-keys"
Jul 17 10:21:48.914991 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:48.914793    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="de4934ee-ae73-4f9f-8747-cdcd88c3dfbe" containerName="revoke-keys"
Jul 17 10:21:48.914991 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:48.914809    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="de23c962-b159-4716-b9ed-c12a8bd92adf" containerName="revoke-keys"
Jul 17 10:21:48.914991 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:48.914819    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="de4934ee-ae73-4f9f-8747-cdcd88c3dfbe" containerName="revoke-keys"
Jul 17 10:21:48.914991 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:48.914827    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="de4934ee-ae73-4f9f-8747-cdcd88c3dfbe" containerName="revoke-keys"
Jul 17 10:21:48.914991 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:48.914837    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="de23c962-b159-4716-b9ed-c12a8bd92adf" containerName="revoke-keys"
Jul 17 10:21:48.914991 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:48.914845    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="de23c962-b159-4716-b9ed-c12a8bd92adf" containerName="revoke-keys"
Jul 17 10:21:48.919683 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:48.919654    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wpj4r"
Jul 17 10:21:48.922839 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:48.922810    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:21:48.924591 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:48.924558    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wpj4r"]
Jul 17 10:21:49.012027 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:49.011980    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cdbb542-2880-4aaf-8b68-6ca256ce0ef4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-wpj4r\" (UID: \"1cdbb542-2880-4aaf-8b68-6ca256ce0ef4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wpj4r"
Jul 17 10:21:49.113017 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:49.112974    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cdbb542-2880-4aaf-8b68-6ca256ce0ef4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-wpj4r\" (UID: \"1cdbb542-2880-4aaf-8b68-6ca256ce0ef4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wpj4r"
Jul 17 10:21:49.113611 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:49.113592    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cdbb542-2880-4aaf-8b68-6ca256ce0ef4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-wpj4r\" (UID: \"1cdbb542-2880-4aaf-8b68-6ca256ce0ef4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wpj4r"
Jul 17 10:21:49.234471 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:49.234371    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wpj4r"
Jul 17 10:21:49.392509 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:49.392479    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wpj4r"]
Jul 17 10:21:49.394182 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:21:49.394158    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cdbb542_2880_4aaf_8b68_6ca256ce0ef4.slice/crio-4862ad700b4fb0900b073929bc5e911547ce96dc1477083e877c22cb0de43177 WatchSource:0}: Error finding container 4862ad700b4fb0900b073929bc5e911547ce96dc1477083e877c22cb0de43177: Status 404 returned error can't find the container with id 4862ad700b4fb0900b073929bc5e911547ce96dc1477083e877c22cb0de43177
Jul 17 10:21:50.109342 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:50.109306    2574 generic.go:356] "Generic (PLEG): container finished" podID="1cdbb542-2880-4aaf-8b68-6ca256ce0ef4" containerID="b802b48c6177ed26108a4f4734c7002a33e5bff2d05fe79d1b56cedc7f911c7f" exitCode=6
Jul 17 10:21:50.109791 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:50.109395    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wpj4r" event={"ID":"1cdbb542-2880-4aaf-8b68-6ca256ce0ef4","Type":"ContainerDied","Data":"b802b48c6177ed26108a4f4734c7002a33e5bff2d05fe79d1b56cedc7f911c7f"}
Jul 17 10:21:50.109791 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:50.109465    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wpj4r" event={"ID":"1cdbb542-2880-4aaf-8b68-6ca256ce0ef4","Type":"ContainerStarted","Data":"4862ad700b4fb0900b073929bc5e911547ce96dc1477083e877c22cb0de43177"}
Jul 17 10:21:50.109791 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:50.109610    2574 scope.go:117] "RemoveContainer" containerID="b802b48c6177ed26108a4f4734c7002a33e5bff2d05fe79d1b56cedc7f911c7f"
Jul 17 10:21:51.118436 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:51.118399    2574 generic.go:356] "Generic (PLEG): container finished" podID="1cdbb542-2880-4aaf-8b68-6ca256ce0ef4" containerID="ef55a5a7b57c40b19ffc46007f468a0f35604a27b1493e1cba541d5f76862437" exitCode=6
Jul 17 10:21:51.118867 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:51.118475    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wpj4r" event={"ID":"1cdbb542-2880-4aaf-8b68-6ca256ce0ef4","Type":"ContainerDied","Data":"ef55a5a7b57c40b19ffc46007f468a0f35604a27b1493e1cba541d5f76862437"}
Jul 17 10:21:51.118867 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:51.118542    2574 scope.go:117] "RemoveContainer" containerID="b802b48c6177ed26108a4f4734c7002a33e5bff2d05fe79d1b56cedc7f911c7f"
Jul 17 10:21:51.118867 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:51.118824    2574 scope.go:117] "RemoveContainer" containerID="ef55a5a7b57c40b19ffc46007f468a0f35604a27b1493e1cba541d5f76862437"
Jul 17 10:21:51.119071 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:21:51.119047    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-wpj4r_odh-ai-gateway-infra(1cdbb542-2880-4aaf-8b68-6ca256ce0ef4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wpj4r" podUID="1cdbb542-2880-4aaf-8b68-6ca256ce0ef4"
Jul 17 10:21:52.128057 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:52.128032    2574 scope.go:117] "RemoveContainer" containerID="ef55a5a7b57c40b19ffc46007f468a0f35604a27b1493e1cba541d5f76862437"
Jul 17 10:21:52.128500 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:21:52.128257    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-wpj4r_odh-ai-gateway-infra(1cdbb542-2880-4aaf-8b68-6ca256ce0ef4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wpj4r" podUID="1cdbb542-2880-4aaf-8b68-6ca256ce0ef4"
Jul 17 10:21:58.018749 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:58.018708    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-92m7l"]
Jul 17 10:21:58.019293 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:58.019211    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de4934ee-ae73-4f9f-8747-cdcd88c3dfbe" containerName="revoke-keys"
Jul 17 10:21:58.019293 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:58.019224    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="de4934ee-ae73-4f9f-8747-cdcd88c3dfbe" containerName="revoke-keys"
Jul 17 10:21:58.022570 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:58.022551    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-92m7l"
Jul 17 10:21:58.027907 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:58.027874    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-92m7l"]
Jul 17 10:21:58.208020 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:58.207961    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/af4da765-16c1-4c33-aa27-d53b2d6bf53a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-92m7l\" (UID: \"af4da765-16c1-4c33-aa27-d53b2d6bf53a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-92m7l"
Jul 17 10:21:58.309350 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:58.309224    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/af4da765-16c1-4c33-aa27-d53b2d6bf53a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-92m7l\" (UID: \"af4da765-16c1-4c33-aa27-d53b2d6bf53a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-92m7l"
Jul 17 10:21:58.309939 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:58.309912    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/af4da765-16c1-4c33-aa27-d53b2d6bf53a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-5109c499-92m7l\" (UID: \"af4da765-16c1-4c33-aa27-d53b2d6bf53a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-92m7l"
Jul 17 10:21:58.335620 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:58.335583    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-92m7l"
Jul 17 10:21:58.492833 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:58.492803    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-92m7l"]
Jul 17 10:21:58.494491 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:21:58.494462    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf4da765_16c1_4c33_aa27_d53b2d6bf53a.slice/crio-bb0695ec0360260a9cba7f14821875259896d01584da2e71740ef6a8aba8019b WatchSource:0}: Error finding container bb0695ec0360260a9cba7f14821875259896d01584da2e71740ef6a8aba8019b: Status 404 returned error can't find the container with id bb0695ec0360260a9cba7f14821875259896d01584da2e71740ef6a8aba8019b
Jul 17 10:21:59.170309 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:59.170272    2574 generic.go:356] "Generic (PLEG): container finished" podID="af4da765-16c1-4c33-aa27-d53b2d6bf53a" containerID="baa8299a4998d599b26755a6dbf5381f323ba750d3423fd2153db6a88c7baf25" exitCode=6
Jul 17 10:21:59.170729 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:59.170371    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-92m7l" event={"ID":"af4da765-16c1-4c33-aa27-d53b2d6bf53a","Type":"ContainerDied","Data":"baa8299a4998d599b26755a6dbf5381f323ba750d3423fd2153db6a88c7baf25"}
Jul 17 10:21:59.170729 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:59.170422    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-92m7l" event={"ID":"af4da765-16c1-4c33-aa27-d53b2d6bf53a","Type":"ContainerStarted","Data":"bb0695ec0360260a9cba7f14821875259896d01584da2e71740ef6a8aba8019b"}
Jul 17 10:21:59.170729 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:21:59.170578    2574 scope.go:117] "RemoveContainer" containerID="baa8299a4998d599b26755a6dbf5381f323ba750d3423fd2153db6a88c7baf25"
Jul 17 10:22:00.179774 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:00.179735    2574 generic.go:356] "Generic (PLEG): container finished" podID="af4da765-16c1-4c33-aa27-d53b2d6bf53a" containerID="77769cf2ff212ebb931c2ed472c566663893ccb11766895e5908a5759c39424c" exitCode=6
Jul 17 10:22:00.180223 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:00.179783    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-92m7l" event={"ID":"af4da765-16c1-4c33-aa27-d53b2d6bf53a","Type":"ContainerDied","Data":"77769cf2ff212ebb931c2ed472c566663893ccb11766895e5908a5759c39424c"}
Jul 17 10:22:00.180223 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:00.179866    2574 scope.go:117] "RemoveContainer" containerID="baa8299a4998d599b26755a6dbf5381f323ba750d3423fd2153db6a88c7baf25"
Jul 17 10:22:00.180223 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:00.180045    2574 scope.go:117] "RemoveContainer" containerID="77769cf2ff212ebb931c2ed472c566663893ccb11766895e5908a5759c39424c"
Jul 17 10:22:00.180425 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:22:00.180344    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-92m7l_odh-ai-gateway-infra(af4da765-16c1-4c33-aa27-d53b2d6bf53a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-92m7l" podUID="af4da765-16c1-4c33-aa27-d53b2d6bf53a"
Jul 17 10:22:01.187604 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:01.187573    2574 scope.go:117] "RemoveContainer" containerID="77769cf2ff212ebb931c2ed472c566663893ccb11766895e5908a5759c39424c"
Jul 17 10:22:01.188032 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:22:01.187798    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-92m7l_odh-ai-gateway-infra(af4da765-16c1-4c33-aa27-d53b2d6bf53a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-92m7l" podUID="af4da765-16c1-4c33-aa27-d53b2d6bf53a"
Jul 17 10:22:06.287120 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:06.287030    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnksj"]
Jul 17 10:22:06.291996 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:06.291971    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnksj"
Jul 17 10:22:06.297914 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:06.297873    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnksj"]
Jul 17 10:22:06.389968 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:06.389906    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/308b0a62-2142-49df-aa5f-2fd34615ec72-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-gnksj\" (UID: \"308b0a62-2142-49df-aa5f-2fd34615ec72\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnksj"
Jul 17 10:22:06.490986 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:06.490939    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/308b0a62-2142-49df-aa5f-2fd34615ec72-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-gnksj\" (UID: \"308b0a62-2142-49df-aa5f-2fd34615ec72\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnksj"
Jul 17 10:22:06.491634 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:06.491614    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/308b0a62-2142-49df-aa5f-2fd34615ec72-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-gnksj\" (UID: \"308b0a62-2142-49df-aa5f-2fd34615ec72\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnksj"
Jul 17 10:22:06.607191 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:06.607101    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnksj"
Jul 17 10:22:06.762864 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:06.762834    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnksj"]
Jul 17 10:22:06.764835 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:22:06.764795    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod308b0a62_2142_49df_aa5f_2fd34615ec72.slice/crio-f7fa44f96099e87358525fbcc7d72c5cec73cf2159bc28cbaca9584de7f61625 WatchSource:0}: Error finding container f7fa44f96099e87358525fbcc7d72c5cec73cf2159bc28cbaca9584de7f61625: Status 404 returned error can't find the container with id f7fa44f96099e87358525fbcc7d72c5cec73cf2159bc28cbaca9584de7f61625
Jul 17 10:22:06.891915 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:06.891821    2574 scope.go:117] "RemoveContainer" containerID="ef55a5a7b57c40b19ffc46007f468a0f35604a27b1493e1cba541d5f76862437"
Jul 17 10:22:07.224253 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:07.224205    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnksj" event={"ID":"308b0a62-2142-49df-aa5f-2fd34615ec72","Type":"ContainerStarted","Data":"8fb33b3e2e612bd28a5009340828f07fe0131122ed1e55e00a87ea84568ec760"}
Jul 17 10:22:07.224469 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:07.224298    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnksj" event={"ID":"308b0a62-2142-49df-aa5f-2fd34615ec72","Type":"ContainerStarted","Data":"f7fa44f96099e87358525fbcc7d72c5cec73cf2159bc28cbaca9584de7f61625"}
Jul 17 10:22:07.243435 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:07.243377    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnksj" podStartSLOduration=1.243356363 podStartE2EDuration="1.243356363s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:22:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:22:07.240523885 +0000 UTC m=+3324.823162264" watchObservedRunningTime="2026-07-17 10:22:07.243356363 +0000 UTC m=+3324.825994739"
Jul 17 10:22:08.232198 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:08.232159    2574 generic.go:356] "Generic (PLEG): container finished" podID="308b0a62-2142-49df-aa5f-2fd34615ec72" containerID="8fb33b3e2e612bd28a5009340828f07fe0131122ed1e55e00a87ea84568ec760" exitCode=6
Jul 17 10:22:08.232674 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:08.232258    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnksj" event={"ID":"308b0a62-2142-49df-aa5f-2fd34615ec72","Type":"ContainerDied","Data":"8fb33b3e2e612bd28a5009340828f07fe0131122ed1e55e00a87ea84568ec760"}
Jul 17 10:22:08.232674 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:08.232458    2574 scope.go:117] "RemoveContainer" containerID="8fb33b3e2e612bd28a5009340828f07fe0131122ed1e55e00a87ea84568ec760"
Jul 17 10:22:08.234522 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:08.234499    2574 generic.go:356] "Generic (PLEG): container finished" podID="1cdbb542-2880-4aaf-8b68-6ca256ce0ef4" containerID="2e93fbf164186ef5dc78f5bfbe70d0e562d113a5e6945c92cd58d1cdc3ceb60b" exitCode=6
Jul 17 10:22:08.234674 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:08.234571    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wpj4r" event={"ID":"1cdbb542-2880-4aaf-8b68-6ca256ce0ef4","Type":"ContainerDied","Data":"2e93fbf164186ef5dc78f5bfbe70d0e562d113a5e6945c92cd58d1cdc3ceb60b"}
Jul 17 10:22:08.234674 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:08.234628    2574 scope.go:117] "RemoveContainer" containerID="ef55a5a7b57c40b19ffc46007f468a0f35604a27b1493e1cba541d5f76862437"
Jul 17 10:22:08.234926 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:08.234905    2574 scope.go:117] "RemoveContainer" containerID="2e93fbf164186ef5dc78f5bfbe70d0e562d113a5e6945c92cd58d1cdc3ceb60b"
Jul 17 10:22:08.235165 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:22:08.235141    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-wpj4r_odh-ai-gateway-infra(1cdbb542-2880-4aaf-8b68-6ca256ce0ef4)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wpj4r" podUID="1cdbb542-2880-4aaf-8b68-6ca256ce0ef4"
Jul 17 10:22:09.242646 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:09.242609    2574 generic.go:356] "Generic (PLEG): container finished" podID="308b0a62-2142-49df-aa5f-2fd34615ec72" containerID="0e61f60cb8e4f2eed94f6885ece465fb8e74fee580c580b6835d596c5426ecc9" exitCode=6
Jul 17 10:22:09.243131 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:09.242702    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnksj" event={"ID":"308b0a62-2142-49df-aa5f-2fd34615ec72","Type":"ContainerDied","Data":"0e61f60cb8e4f2eed94f6885ece465fb8e74fee580c580b6835d596c5426ecc9"}
Jul 17 10:22:09.243131 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:09.242763    2574 scope.go:117] "RemoveContainer" containerID="8fb33b3e2e612bd28a5009340828f07fe0131122ed1e55e00a87ea84568ec760"
Jul 17 10:22:09.243131 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:09.242936    2574 scope.go:117] "RemoveContainer" containerID="0e61f60cb8e4f2eed94f6885ece465fb8e74fee580c580b6835d596c5426ecc9"
Jul 17 10:22:09.243346 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:22:09.243170    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-gnksj_odh-ai-gateway-infra(308b0a62-2142-49df-aa5f-2fd34615ec72)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnksj" podUID="308b0a62-2142-49df-aa5f-2fd34615ec72"
Jul 17 10:22:09.274365 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:09.274322    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wpj4r"]
Jul 17 10:22:09.445228 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:09.445191    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wpj4r"
Jul 17 10:22:09.523286 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:09.523160    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cdbb542-2880-4aaf-8b68-6ca256ce0ef4-maas-api-service-ca\") pod \"1cdbb542-2880-4aaf-8b68-6ca256ce0ef4\" (UID: \"1cdbb542-2880-4aaf-8b68-6ca256ce0ef4\") "
Jul 17 10:22:09.523579 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:09.523552    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cdbb542-2880-4aaf-8b68-6ca256ce0ef4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1cdbb542-2880-4aaf-8b68-6ca256ce0ef4" (UID: "1cdbb542-2880-4aaf-8b68-6ca256ce0ef4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:22:09.624595 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:09.624555    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cdbb542-2880-4aaf-8b68-6ca256ce0ef4-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:22:10.253531 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:10.253503    2574 scope.go:117] "RemoveContainer" containerID="0e61f60cb8e4f2eed94f6885ece465fb8e74fee580c580b6835d596c5426ecc9"
Jul 17 10:22:10.254000 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:22:10.253780    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-gnksj_odh-ai-gateway-infra(308b0a62-2142-49df-aa5f-2fd34615ec72)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnksj" podUID="308b0a62-2142-49df-aa5f-2fd34615ec72"
Jul 17 10:22:10.254788 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:10.254766    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wpj4r"
Jul 17 10:22:10.254788 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:10.254773    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wpj4r" event={"ID":"1cdbb542-2880-4aaf-8b68-6ca256ce0ef4","Type":"ContainerDied","Data":"4862ad700b4fb0900b073929bc5e911547ce96dc1477083e877c22cb0de43177"}
Jul 17 10:22:10.254973 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:10.254852    2574 scope.go:117] "RemoveContainer" containerID="2e93fbf164186ef5dc78f5bfbe70d0e562d113a5e6945c92cd58d1cdc3ceb60b"
Jul 17 10:22:10.284716 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:10.284675    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wpj4r"]
Jul 17 10:22:10.291032 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:10.290999    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-wpj4r"]
Jul 17 10:22:10.898628 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:10.898588    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cdbb542-2880-4aaf-8b68-6ca256ce0ef4" path="/var/lib/kubelet/pods/1cdbb542-2880-4aaf-8b68-6ca256ce0ef4/volumes"
Jul 17 10:22:14.891870 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:14.891830    2574 scope.go:117] "RemoveContainer" containerID="77769cf2ff212ebb931c2ed472c566663893ccb11766895e5908a5759c39424c"
Jul 17 10:22:15.288283 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:15.288223    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-92m7l" event={"ID":"af4da765-16c1-4c33-aa27-d53b2d6bf53a","Type":"ContainerStarted","Data":"9e58fe07fb6c09de36aa9740cf2d2a482ab3a649d4c99ac713955b29be33a607"}
Jul 17 10:22:15.305607 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:15.305552    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-92m7l" podStartSLOduration=17.305536202 podStartE2EDuration="17.305536202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:21:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:22:15.304317142 +0000 UTC m=+3332.886955519" watchObservedRunningTime="2026-07-17 10:22:15.305536202 +0000 UTC m=+3332.888174601"
Jul 17 10:22:16.295558 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:16.295521    2574 generic.go:356] "Generic (PLEG): container finished" podID="af4da765-16c1-4c33-aa27-d53b2d6bf53a" containerID="9e58fe07fb6c09de36aa9740cf2d2a482ab3a649d4c99ac713955b29be33a607" exitCode=6
Jul 17 10:22:16.296015 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:16.295593    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-92m7l" event={"ID":"af4da765-16c1-4c33-aa27-d53b2d6bf53a","Type":"ContainerDied","Data":"9e58fe07fb6c09de36aa9740cf2d2a482ab3a649d4c99ac713955b29be33a607"}
Jul 17 10:22:16.296015 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:16.295659    2574 scope.go:117] "RemoveContainer" containerID="77769cf2ff212ebb931c2ed472c566663893ccb11766895e5908a5759c39424c"
Jul 17 10:22:16.296015 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:16.295943    2574 scope.go:117] "RemoveContainer" containerID="9e58fe07fb6c09de36aa9740cf2d2a482ab3a649d4c99ac713955b29be33a607"
Jul 17 10:22:16.296264 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:22:16.296216    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-5109c499-92m7l_odh-ai-gateway-infra(af4da765-16c1-4c33-aa27-d53b2d6bf53a)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-92m7l" podUID="af4da765-16c1-4c33-aa27-d53b2d6bf53a"
Jul 17 10:22:16.320935 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:16.320896    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-92m7l"]
Jul 17 10:22:17.462500 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:17.462466    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-92m7l"
Jul 17 10:22:17.507876 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:17.507837    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/af4da765-16c1-4c33-aa27-d53b2d6bf53a-maas-api-service-ca\") pod \"af4da765-16c1-4c33-aa27-d53b2d6bf53a\" (UID: \"af4da765-16c1-4c33-aa27-d53b2d6bf53a\") "
Jul 17 10:22:17.508298 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:17.508275    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af4da765-16c1-4c33-aa27-d53b2d6bf53a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "af4da765-16c1-4c33-aa27-d53b2d6bf53a" (UID: "af4da765-16c1-4c33-aa27-d53b2d6bf53a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:22:17.609602 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:17.609497    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/af4da765-16c1-4c33-aa27-d53b2d6bf53a-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:22:18.312867 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:18.312824    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-92m7l" event={"ID":"af4da765-16c1-4c33-aa27-d53b2d6bf53a","Type":"ContainerDied","Data":"bb0695ec0360260a9cba7f14821875259896d01584da2e71740ef6a8aba8019b"}
Jul 17 10:22:18.312867 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:18.312863    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-92m7l"
Jul 17 10:22:18.313104 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:18.312907    2574 scope.go:117] "RemoveContainer" containerID="9e58fe07fb6c09de36aa9740cf2d2a482ab3a649d4c99ac713955b29be33a607"
Jul 17 10:22:18.339957 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:18.339922    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-92m7l"]
Jul 17 10:22:18.345017 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:18.344984    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-5109c499-92m7l"]
Jul 17 10:22:18.898636 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:18.898591    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af4da765-16c1-4c33-aa27-d53b2d6bf53a" path="/var/lib/kubelet/pods/af4da765-16c1-4c33-aa27-d53b2d6bf53a/volumes"
Jul 17 10:22:23.892088 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:23.892044    2574 scope.go:117] "RemoveContainer" containerID="0e61f60cb8e4f2eed94f6885ece465fb8e74fee580c580b6835d596c5426ecc9"
Jul 17 10:22:24.350694 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:24.350660    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnksj" event={"ID":"308b0a62-2142-49df-aa5f-2fd34615ec72","Type":"ContainerStarted","Data":"a6db4a0b55c638b78f0604d9eeaa4e3ffa72fa33dd73d40bfa35554a81413a1f"}
Jul 17 10:22:25.358181 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:25.358145    2574 generic.go:356] "Generic (PLEG): container finished" podID="308b0a62-2142-49df-aa5f-2fd34615ec72" containerID="a6db4a0b55c638b78f0604d9eeaa4e3ffa72fa33dd73d40bfa35554a81413a1f" exitCode=6
Jul 17 10:22:25.358667 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:25.358219    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnksj" event={"ID":"308b0a62-2142-49df-aa5f-2fd34615ec72","Type":"ContainerDied","Data":"a6db4a0b55c638b78f0604d9eeaa4e3ffa72fa33dd73d40bfa35554a81413a1f"}
Jul 17 10:22:25.358667 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:25.358308    2574 scope.go:117] "RemoveContainer" containerID="0e61f60cb8e4f2eed94f6885ece465fb8e74fee580c580b6835d596c5426ecc9"
Jul 17 10:22:25.358667 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:25.358604    2574 scope.go:117] "RemoveContainer" containerID="a6db4a0b55c638b78f0604d9eeaa4e3ffa72fa33dd73d40bfa35554a81413a1f"
Jul 17 10:22:25.358865 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:22:25.358841    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-gnksj_odh-ai-gateway-infra(308b0a62-2142-49df-aa5f-2fd34615ec72)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnksj" podUID="308b0a62-2142-49df-aa5f-2fd34615ec72"
Jul 17 10:22:25.377632 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:25.377592    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnksj"]
Jul 17 10:22:26.522235 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:26.522198    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnksj"
Jul 17 10:22:26.599894 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:26.599856    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/308b0a62-2142-49df-aa5f-2fd34615ec72-maas-api-service-ca\") pod \"308b0a62-2142-49df-aa5f-2fd34615ec72\" (UID: \"308b0a62-2142-49df-aa5f-2fd34615ec72\") "
Jul 17 10:22:26.600232 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:26.600208    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/308b0a62-2142-49df-aa5f-2fd34615ec72-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "308b0a62-2142-49df-aa5f-2fd34615ec72" (UID: "308b0a62-2142-49df-aa5f-2fd34615ec72"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:22:26.700892 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:26.700844    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/308b0a62-2142-49df-aa5f-2fd34615ec72-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:22:27.373158 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:27.373119    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnksj" event={"ID":"308b0a62-2142-49df-aa5f-2fd34615ec72","Type":"ContainerDied","Data":"f7fa44f96099e87358525fbcc7d72c5cec73cf2159bc28cbaca9584de7f61625"}
Jul 17 10:22:27.373376 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:27.373183    2574 scope.go:117] "RemoveContainer" containerID="a6db4a0b55c638b78f0604d9eeaa4e3ffa72fa33dd73d40bfa35554a81413a1f"
Jul 17 10:22:27.373376 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:27.373136    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnksj"
Jul 17 10:22:27.391541 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:27.391503    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnksj"]
Jul 17 10:22:27.402268 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:27.402207    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-gnksj"]
Jul 17 10:22:28.898418 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:28.898378    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="308b0a62-2142-49df-aa5f-2fd34615ec72" path="/var/lib/kubelet/pods/308b0a62-2142-49df-aa5f-2fd34615ec72/volumes"
Jul 17 10:22:36.826995 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:36.826959    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-fwvdc"]
Jul 17 10:22:36.827477 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:36.827463    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af4da765-16c1-4c33-aa27-d53b2d6bf53a" containerName="revoke-keys"
Jul 17 10:22:36.827524 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:36.827479    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="af4da765-16c1-4c33-aa27-d53b2d6bf53a" containerName="revoke-keys"
Jul 17 10:22:36.827524 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:36.827487    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="308b0a62-2142-49df-aa5f-2fd34615ec72" containerName="revoke-keys"
Jul 17 10:22:36.827524 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:36.827492    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="308b0a62-2142-49df-aa5f-2fd34615ec72" containerName="revoke-keys"
Jul 17 10:22:36.827524 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:36.827500    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af4da765-16c1-4c33-aa27-d53b2d6bf53a" containerName="revoke-keys"
Jul 17 10:22:36.827524 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:36.827505    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="af4da765-16c1-4c33-aa27-d53b2d6bf53a" containerName="revoke-keys"
Jul 17 10:22:36.827524 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:36.827513    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cdbb542-2880-4aaf-8b68-6ca256ce0ef4" containerName="revoke-keys"
Jul 17 10:22:36.827524 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:36.827519    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cdbb542-2880-4aaf-8b68-6ca256ce0ef4" containerName="revoke-keys"
Jul 17 10:22:36.827524 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:36.827526    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cdbb542-2880-4aaf-8b68-6ca256ce0ef4" containerName="revoke-keys"
Jul 17 10:22:36.827777 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:36.827531    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cdbb542-2880-4aaf-8b68-6ca256ce0ef4" containerName="revoke-keys"
Jul 17 10:22:36.827777 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:36.827538    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="308b0a62-2142-49df-aa5f-2fd34615ec72" containerName="revoke-keys"
Jul 17 10:22:36.827777 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:36.827543    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="308b0a62-2142-49df-aa5f-2fd34615ec72" containerName="revoke-keys"
Jul 17 10:22:36.827777 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:36.827561    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af4da765-16c1-4c33-aa27-d53b2d6bf53a" containerName="revoke-keys"
Jul 17 10:22:36.827777 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:36.827566    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="af4da765-16c1-4c33-aa27-d53b2d6bf53a" containerName="revoke-keys"
Jul 17 10:22:36.827777 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:36.827575    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="308b0a62-2142-49df-aa5f-2fd34615ec72" containerName="revoke-keys"
Jul 17 10:22:36.827777 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:36.827580    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="308b0a62-2142-49df-aa5f-2fd34615ec72" containerName="revoke-keys"
Jul 17 10:22:36.827777 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:36.827646    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cdbb542-2880-4aaf-8b68-6ca256ce0ef4" containerName="revoke-keys"
Jul 17 10:22:36.827777 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:36.827653    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cdbb542-2880-4aaf-8b68-6ca256ce0ef4" containerName="revoke-keys"
Jul 17 10:22:36.827777 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:36.827659    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="af4da765-16c1-4c33-aa27-d53b2d6bf53a" containerName="revoke-keys"
Jul 17 10:22:36.827777 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:36.827664    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="308b0a62-2142-49df-aa5f-2fd34615ec72" containerName="revoke-keys"
Jul 17 10:22:36.827777 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:36.827670    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="308b0a62-2142-49df-aa5f-2fd34615ec72" containerName="revoke-keys"
Jul 17 10:22:36.827777 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:36.827676    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="308b0a62-2142-49df-aa5f-2fd34615ec72" containerName="revoke-keys"
Jul 17 10:22:36.827777 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:36.827682    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cdbb542-2880-4aaf-8b68-6ca256ce0ef4" containerName="revoke-keys"
Jul 17 10:22:36.827777 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:36.827689    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="af4da765-16c1-4c33-aa27-d53b2d6bf53a" containerName="revoke-keys"
Jul 17 10:22:36.832369 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:36.832344    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-fwvdc"
Jul 17 10:22:36.835594 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:36.835561    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:22:36.838091 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:36.838064    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-fwvdc"]
Jul 17 10:22:37.006109 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:37.006064    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db0a1d18-6c8b-45de-bb7d-fd10a56367ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-fwvdc\" (UID: \"db0a1d18-6c8b-45de-bb7d-fd10a56367ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-fwvdc"
Jul 17 10:22:37.107385 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:37.107282    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db0a1d18-6c8b-45de-bb7d-fd10a56367ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-fwvdc\" (UID: \"db0a1d18-6c8b-45de-bb7d-fd10a56367ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-fwvdc"
Jul 17 10:22:37.108107 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:37.108079    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db0a1d18-6c8b-45de-bb7d-fd10a56367ab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-fwvdc\" (UID: \"db0a1d18-6c8b-45de-bb7d-fd10a56367ab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-fwvdc"
Jul 17 10:22:37.146283 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:37.146215    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-fwvdc"
Jul 17 10:22:37.298887 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:37.298862    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-fwvdc"]
Jul 17 10:22:37.301550 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:22:37.301508    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb0a1d18_6c8b_45de_bb7d_fd10a56367ab.slice/crio-896243039bfda8e92d29ec49e56e42f8fff6e5686e9e42861908477e5a71e39b WatchSource:0}: Error finding container 896243039bfda8e92d29ec49e56e42f8fff6e5686e9e42861908477e5a71e39b: Status 404 returned error can't find the container with id 896243039bfda8e92d29ec49e56e42f8fff6e5686e9e42861908477e5a71e39b
Jul 17 10:22:37.436031 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:37.435988    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-fwvdc" event={"ID":"db0a1d18-6c8b-45de-bb7d-fd10a56367ab","Type":"ContainerStarted","Data":"896243039bfda8e92d29ec49e56e42f8fff6e5686e9e42861908477e5a71e39b"}
Jul 17 10:22:38.443593 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:38.443550    2574 generic.go:356] "Generic (PLEG): container finished" podID="db0a1d18-6c8b-45de-bb7d-fd10a56367ab" containerID="fc2d26388fcc1f180aabb2a470f0d69ab010bbfc45d679289b752b229d271a15" exitCode=6
Jul 17 10:22:38.443983 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:38.443605    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-fwvdc" event={"ID":"db0a1d18-6c8b-45de-bb7d-fd10a56367ab","Type":"ContainerDied","Data":"fc2d26388fcc1f180aabb2a470f0d69ab010bbfc45d679289b752b229d271a15"}
Jul 17 10:22:38.443983 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:38.443822    2574 scope.go:117] "RemoveContainer" containerID="fc2d26388fcc1f180aabb2a470f0d69ab010bbfc45d679289b752b229d271a15"
Jul 17 10:22:39.452026 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:39.451992    2574 generic.go:356] "Generic (PLEG): container finished" podID="db0a1d18-6c8b-45de-bb7d-fd10a56367ab" containerID="d0896dbdaf240d51d96bde7796a633a984ea84a4cdfcb3e3801121ee48b4fcfd" exitCode=6
Jul 17 10:22:39.452536 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:39.452035    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-fwvdc" event={"ID":"db0a1d18-6c8b-45de-bb7d-fd10a56367ab","Type":"ContainerDied","Data":"d0896dbdaf240d51d96bde7796a633a984ea84a4cdfcb3e3801121ee48b4fcfd"}
Jul 17 10:22:39.452536 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:39.452123    2574 scope.go:117] "RemoveContainer" containerID="fc2d26388fcc1f180aabb2a470f0d69ab010bbfc45d679289b752b229d271a15"
Jul 17 10:22:39.452536 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:39.452446    2574 scope.go:117] "RemoveContainer" containerID="d0896dbdaf240d51d96bde7796a633a984ea84a4cdfcb3e3801121ee48b4fcfd"
Jul 17 10:22:39.452733 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:22:39.452713    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-fwvdc_odh-ai-gateway-infra(db0a1d18-6c8b-45de-bb7d-fd10a56367ab)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-fwvdc" podUID="db0a1d18-6c8b-45de-bb7d-fd10a56367ab"
Jul 17 10:22:40.459905 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:40.459873    2574 scope.go:117] "RemoveContainer" containerID="d0896dbdaf240d51d96bde7796a633a984ea84a4cdfcb3e3801121ee48b4fcfd"
Jul 17 10:22:40.460436 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:22:40.460161    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-fwvdc_odh-ai-gateway-infra(db0a1d18-6c8b-45de-bb7d-fd10a56367ab)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-fwvdc" podUID="db0a1d18-6c8b-45de-bb7d-fd10a56367ab"
Jul 17 10:22:52.895212 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:52.895175    2574 scope.go:117] "RemoveContainer" containerID="d0896dbdaf240d51d96bde7796a633a984ea84a4cdfcb3e3801121ee48b4fcfd"
Jul 17 10:22:53.541902 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:53.541865    2574 generic.go:356] "Generic (PLEG): container finished" podID="db0a1d18-6c8b-45de-bb7d-fd10a56367ab" containerID="84e41519fac33673de652eea2e986d424d90836d4f0b50c2ee7aafebc5d3b275" exitCode=6
Jul 17 10:22:53.542082 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:53.541925    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-fwvdc" event={"ID":"db0a1d18-6c8b-45de-bb7d-fd10a56367ab","Type":"ContainerDied","Data":"84e41519fac33673de652eea2e986d424d90836d4f0b50c2ee7aafebc5d3b275"}
Jul 17 10:22:53.542082 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:53.541995    2574 scope.go:117] "RemoveContainer" containerID="d0896dbdaf240d51d96bde7796a633a984ea84a4cdfcb3e3801121ee48b4fcfd"
Jul 17 10:22:53.542391 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:53.542372    2574 scope.go:117] "RemoveContainer" containerID="84e41519fac33673de652eea2e986d424d90836d4f0b50c2ee7aafebc5d3b275"
Jul 17 10:22:53.542641 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:22:53.542622    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-fwvdc_odh-ai-gateway-infra(db0a1d18-6c8b-45de-bb7d-fd10a56367ab)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-fwvdc" podUID="db0a1d18-6c8b-45de-bb7d-fd10a56367ab"
Jul 17 10:22:54.572391 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:54.572344    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-fwvdc"]
Jul 17 10:22:54.737881 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:54.737844    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-fwvdc"
Jul 17 10:22:54.793665 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:54.793622    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db0a1d18-6c8b-45de-bb7d-fd10a56367ab-maas-api-service-ca\") pod \"db0a1d18-6c8b-45de-bb7d-fd10a56367ab\" (UID: \"db0a1d18-6c8b-45de-bb7d-fd10a56367ab\") "
Jul 17 10:22:54.794076 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:54.794051    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db0a1d18-6c8b-45de-bb7d-fd10a56367ab-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "db0a1d18-6c8b-45de-bb7d-fd10a56367ab" (UID: "db0a1d18-6c8b-45de-bb7d-fd10a56367ab"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:22:54.894534 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:54.894441    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/db0a1d18-6c8b-45de-bb7d-fd10a56367ab-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:22:55.557605 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:55.557575    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-fwvdc"
Jul 17 10:22:55.557834 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:55.557569    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-fwvdc" event={"ID":"db0a1d18-6c8b-45de-bb7d-fd10a56367ab","Type":"ContainerDied","Data":"896243039bfda8e92d29ec49e56e42f8fff6e5686e9e42861908477e5a71e39b"}
Jul 17 10:22:55.557834 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:55.557710    2574 scope.go:117] "RemoveContainer" containerID="84e41519fac33673de652eea2e986d424d90836d4f0b50c2ee7aafebc5d3b275"
Jul 17 10:22:55.578062 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:55.578025    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-fwvdc"]
Jul 17 10:22:55.585796 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:55.585755    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-fwvdc"]
Jul 17 10:22:56.898501 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:22:56.898462    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db0a1d18-6c8b-45de-bb7d-fd10a56367ab" path="/var/lib/kubelet/pods/db0a1d18-6c8b-45de-bb7d-fd10a56367ab/volumes"
Jul 17 10:23:43.777016 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:43.776979    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"]
Jul 17 10:23:43.777600 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:43.777298    2574 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp" podUID="cf340eb4-e72a-4fe8-b125-158eb16e0543" containerName="istio-proxy" containerID="cri-o://10d0d7feda4aa2787e0cbf957c81051939ba0a7f11214784853e39b11d2620aa" gracePeriod=30
Jul 17 10:23:46.918174 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:46.918127    2574 patch_prober.go:28] interesting pod/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 17 10:23:46.918608 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:46.918200    2574 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp" podUID="cf340eb4-e72a-4fe8-b125-158eb16e0543" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 17 10:23:49.069311 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.069277    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:23:49.254977 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.254938    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf340eb4-e72a-4fe8-b125-158eb16e0543-istio-token\") pod \"cf340eb4-e72a-4fe8-b125-158eb16e0543\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") "
Jul 17 10:23:49.254977 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.254980    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf340eb4-e72a-4fe8-b125-158eb16e0543-istio-podinfo\") pod \"cf340eb4-e72a-4fe8-b125-158eb16e0543\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") "
Jul 17 10:23:49.255283 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.255044    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-istio-envoy\") pod \"cf340eb4-e72a-4fe8-b125-158eb16e0543\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") "
Jul 17 10:23:49.255283 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.255147    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-istio-data\") pod \"cf340eb4-e72a-4fe8-b125-158eb16e0543\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") "
Jul 17 10:23:49.255283 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.255171    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf340eb4-e72a-4fe8-b125-158eb16e0543-istiod-ca-cert\") pod \"cf340eb4-e72a-4fe8-b125-158eb16e0543\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") "
Jul 17 10:23:49.255283 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.255214    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-workload-socket\") pod \"cf340eb4-e72a-4fe8-b125-158eb16e0543\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") "
Jul 17 10:23:49.255283 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.255278    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-workload-certs\") pod \"cf340eb4-e72a-4fe8-b125-158eb16e0543\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") "
Jul 17 10:23:49.255548 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.255302    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-credential-socket\") pod \"cf340eb4-e72a-4fe8-b125-158eb16e0543\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") "
Jul 17 10:23:49.255548 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.255349    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qllph\" (UniqueName: \"kubernetes.io/projected/cf340eb4-e72a-4fe8-b125-158eb16e0543-kube-api-access-qllph\") pod \"cf340eb4-e72a-4fe8-b125-158eb16e0543\" (UID: \"cf340eb4-e72a-4fe8-b125-158eb16e0543\") "
Jul 17 10:23:49.255649 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.255619    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf340eb4-e72a-4fe8-b125-158eb16e0543-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cf340eb4-e72a-4fe8-b125-158eb16e0543" (UID: "cf340eb4-e72a-4fe8-b125-158eb16e0543"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:23:49.255933 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.255755    2574 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf340eb4-e72a-4fe8-b125-158eb16e0543-istiod-ca-cert\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:23:49.255933 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.255630    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cf340eb4-e72a-4fe8-b125-158eb16e0543" (UID: "cf340eb4-e72a-4fe8-b125-158eb16e0543"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:23:49.255933 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.255779    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cf340eb4-e72a-4fe8-b125-158eb16e0543" (UID: "cf340eb4-e72a-4fe8-b125-158eb16e0543"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:23:49.255933 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.255890    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cf340eb4-e72a-4fe8-b125-158eb16e0543" (UID: "cf340eb4-e72a-4fe8-b125-158eb16e0543"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:23:49.256260 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.256214    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-istio-data" (OuterVolumeSpecName: "istio-data") pod "cf340eb4-e72a-4fe8-b125-158eb16e0543" (UID: "cf340eb4-e72a-4fe8-b125-158eb16e0543"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:23:49.257571 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.257546    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cf340eb4-e72a-4fe8-b125-158eb16e0543" (UID: "cf340eb4-e72a-4fe8-b125-158eb16e0543"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 17 10:23:49.257677 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.257567    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf340eb4-e72a-4fe8-b125-158eb16e0543-istio-token" (OuterVolumeSpecName: "istio-token") pod "cf340eb4-e72a-4fe8-b125-158eb16e0543" (UID: "cf340eb4-e72a-4fe8-b125-158eb16e0543"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:23:49.258169 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.258147    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cf340eb4-e72a-4fe8-b125-158eb16e0543-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cf340eb4-e72a-4fe8-b125-158eb16e0543" (UID: "cf340eb4-e72a-4fe8-b125-158eb16e0543"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 17 10:23:49.258169 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.258154    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf340eb4-e72a-4fe8-b125-158eb16e0543-kube-api-access-qllph" (OuterVolumeSpecName: "kube-api-access-qllph") pod "cf340eb4-e72a-4fe8-b125-158eb16e0543" (UID: "cf340eb4-e72a-4fe8-b125-158eb16e0543"). InnerVolumeSpecName "kube-api-access-qllph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 17 10:23:49.356783 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.356689    2574 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-istio-data\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:23:49.356783 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.356724    2574 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-workload-socket\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:23:49.356783 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.356733    2574 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-workload-certs\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:23:49.356783 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.356741    2574 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-credential-socket\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:23:49.356783 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.356751    2574 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qllph\" (UniqueName: \"kubernetes.io/projected/cf340eb4-e72a-4fe8-b125-158eb16e0543-kube-api-access-qllph\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:23:49.356783 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.356759    2574 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf340eb4-e72a-4fe8-b125-158eb16e0543-istio-token\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:23:49.356783 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.356768    2574 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf340eb4-e72a-4fe8-b125-158eb16e0543-istio-podinfo\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:23:49.356783 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.356775    2574 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf340eb4-e72a-4fe8-b125-158eb16e0543-istio-envoy\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:23:49.886378 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.886338    2574 generic.go:356] "Generic (PLEG): container finished" podID="cf340eb4-e72a-4fe8-b125-158eb16e0543" containerID="10d0d7feda4aa2787e0cbf957c81051939ba0a7f11214784853e39b11d2620aa" exitCode=0
Jul 17 10:23:49.886603 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.886387    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"
Jul 17 10:23:49.886603 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.886394    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp" event={"ID":"cf340eb4-e72a-4fe8-b125-158eb16e0543","Type":"ContainerDied","Data":"10d0d7feda4aa2787e0cbf957c81051939ba0a7f11214784853e39b11d2620aa"}
Jul 17 10:23:49.886603 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.886448    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp" event={"ID":"cf340eb4-e72a-4fe8-b125-158eb16e0543","Type":"ContainerDied","Data":"40c01f3f101af8b53bface510179e81ccad91da5a79b6f44f33df5af5ee9d43a"}
Jul 17 10:23:49.886603 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.886480    2574 scope.go:117] "RemoveContainer" containerID="10d0d7feda4aa2787e0cbf957c81051939ba0a7f11214784853e39b11d2620aa"
Jul 17 10:23:49.896580 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.896560    2574 scope.go:117] "RemoveContainer" containerID="fd4d15df6598ef507cab8c7870cc4442187ae80b71f56cafb78510e19306c226"
Jul 17 10:23:49.905895 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.905874    2574 scope.go:117] "RemoveContainer" containerID="10d0d7feda4aa2787e0cbf957c81051939ba0a7f11214784853e39b11d2620aa"
Jul 17 10:23:49.906253 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:23:49.906217    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10d0d7feda4aa2787e0cbf957c81051939ba0a7f11214784853e39b11d2620aa\": container with ID starting with 10d0d7feda4aa2787e0cbf957c81051939ba0a7f11214784853e39b11d2620aa not found: ID does not exist" containerID="10d0d7feda4aa2787e0cbf957c81051939ba0a7f11214784853e39b11d2620aa"
Jul 17 10:23:49.906308 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.906270    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10d0d7feda4aa2787e0cbf957c81051939ba0a7f11214784853e39b11d2620aa"} err="failed to get container status \"10d0d7feda4aa2787e0cbf957c81051939ba0a7f11214784853e39b11d2620aa\": rpc error: code = NotFound desc = could not find container \"10d0d7feda4aa2787e0cbf957c81051939ba0a7f11214784853e39b11d2620aa\": container with ID starting with 10d0d7feda4aa2787e0cbf957c81051939ba0a7f11214784853e39b11d2620aa not found: ID does not exist"
Jul 17 10:23:49.906308 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.906290    2574 scope.go:117] "RemoveContainer" containerID="fd4d15df6598ef507cab8c7870cc4442187ae80b71f56cafb78510e19306c226"
Jul 17 10:23:49.906567 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:23:49.906544    2574 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd4d15df6598ef507cab8c7870cc4442187ae80b71f56cafb78510e19306c226\": container with ID starting with fd4d15df6598ef507cab8c7870cc4442187ae80b71f56cafb78510e19306c226 not found: ID does not exist" containerID="fd4d15df6598ef507cab8c7870cc4442187ae80b71f56cafb78510e19306c226"
Jul 17 10:23:49.906633 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.906577    2574 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd4d15df6598ef507cab8c7870cc4442187ae80b71f56cafb78510e19306c226"} err="failed to get container status \"fd4d15df6598ef507cab8c7870cc4442187ae80b71f56cafb78510e19306c226\": rpc error: code = NotFound desc = could not find container \"fd4d15df6598ef507cab8c7870cc4442187ae80b71f56cafb78510e19306c226\": container with ID starting with fd4d15df6598ef507cab8c7870cc4442187ae80b71f56cafb78510e19306c226 not found: ID does not exist"
Jul 17 10:23:49.910188 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.910159    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"]
Jul 17 10:23:49.914114 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:49.914088    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-af80d9-openshift-default-ccbd6dd7-gz8qp"]
Jul 17 10:23:50.260452 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:50.260421    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-24c6h_75da319e-cb7b-471b-811c-cf9f9fbf120a/manager/0.log"
Jul 17 10:23:50.389412 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:50.389372    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-v9rhs_03e554d1-089a-4801-aa0c-ace3e09085a2/manager/0.log"
Jul 17 10:23:50.516916 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:50.516835    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6c5c6b98f-rlrg6_8ab844e0-7c94-4003-9e42-ada02b05d4e9/manager/0.log"
Jul 17 10:23:50.642682 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:50.642658    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-96757_e56c1862-83dd-417d-bd30-60abc092e276/server/0.log"
Jul 17 10:23:50.772103 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:50.772023    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-bs5g8_2cdf35a8-c7a2-49c1-98d5-7153f317c517/manager/0.log"
Jul 17 10:23:50.897803 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:50.897762    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf340eb4-e72a-4fe8-b125-158eb16e0543" path="/var/lib/kubelet/pods/cf340eb4-e72a-4fe8-b125-158eb16e0543/volumes"
Jul 17 10:23:51.170060 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:51.170031    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b5566fdf9-ndp9h_ce30ed03-36b2-442a-8dba-13d434d42d39/copy-manifests/0.log"
Jul 17 10:23:51.183577 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:51.183540    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b5566fdf9-ndp9h_ce30ed03-36b2-442a-8dba-13d434d42d39/manager/0.log"
Jul 17 10:23:51.723003 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:51.722968    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29738025-npjpr_e1e47955-752c-4b2e-b76f-9d1efaa72250/cleanup/0.log"
Jul 17 10:23:51.839357 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:51.839319    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29738040-vgggn_5876f95d-d924-4f5f-acaa-8413677c0013/cleanup/0.log"
Jul 17 10:23:52.448507 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:52.448478    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-a-21e67a-fqs97_c29a17d5-9aea-4e31-95ed-29f0b338f3f4/revoke-keys/0.log"
Jul 17 10:23:52.572842 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:52.572813    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-inf-b-1c996b-xms8l_63a05154-b1a1-4aa8-a750-fe842b3b139d/revoke-keys/0.log"
Jul 17 10:23:52.807202 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:52.807128    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-6bff328f-lk45q_0d7247e2-690f-4fa1-858a-b6d1c2d98f43/revoke-keys/0.log"
Jul 17 10:23:52.936099 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:52.936068    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-cda038a0-fcqbz_f61c2442-1e0c-4943-b032-e8ff24d3eeca/revoke-keys/0.log"
Jul 17 10:23:53.290968 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:53.290938    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-lrl7q_645866b9-eee2-4b8c-b3cd-131e8feff4ae/postgres/0.log"
Jul 17 10:23:54.761648 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:54.761616    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-mk4bq_8e21489e-210e-4b51-9777-e637fb5cfc74/manager/0.log"
Jul 17 10:23:54.881560 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:54.881529    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-zvqp9_0e5c29c3-e1f5-48b8-a3a1-f3579196c61c/manager/0.log"
Jul 17 10:23:55.125967 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:55.125887    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-5x7hd_87acf608-ce98-43fc-820d-11da3b3cbc0d/registry-server/0.log"
Jul 17 10:23:55.253744 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:55.253711    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-k8fhl_bcd494e5-0002-47a1-9c6d-c31af2e510e2/manager/0.log"
Jul 17 10:23:55.513646 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:55.513610    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-hrbff_7eceeaf3-ab95-4000-913c-2254baf18b00/manager/0.log"
Jul 17 10:23:56.284777 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.284749    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-546bc49978-vw2hd_1f7ee6c1-0dfe-4612-93a9-beccaa966946/kube-auth-proxy/0.log"
Jul 17 10:23:56.377317 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.377284    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vkpq8"]
Jul 17 10:23:56.377729 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.377716    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cdbb542-2880-4aaf-8b68-6ca256ce0ef4" containerName="revoke-keys"
Jul 17 10:23:56.377784 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.377731    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cdbb542-2880-4aaf-8b68-6ca256ce0ef4" containerName="revoke-keys"
Jul 17 10:23:56.377784 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.377750    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf340eb4-e72a-4fe8-b125-158eb16e0543" containerName="istio-proxy"
Jul 17 10:23:56.377784 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.377756    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf340eb4-e72a-4fe8-b125-158eb16e0543" containerName="istio-proxy"
Jul 17 10:23:56.377784 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.377775    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf340eb4-e72a-4fe8-b125-158eb16e0543" containerName="istio-proxy"
Jul 17 10:23:56.377784 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.377781    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf340eb4-e72a-4fe8-b125-158eb16e0543" containerName="istio-proxy"
Jul 17 10:23:56.377943 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.377789    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db0a1d18-6c8b-45de-bb7d-fd10a56367ab" containerName="revoke-keys"
Jul 17 10:23:56.377943 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.377796    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="db0a1d18-6c8b-45de-bb7d-fd10a56367ab" containerName="revoke-keys"
Jul 17 10:23:56.377943 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.377812    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db0a1d18-6c8b-45de-bb7d-fd10a56367ab" containerName="revoke-keys"
Jul 17 10:23:56.377943 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.377817    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="db0a1d18-6c8b-45de-bb7d-fd10a56367ab" containerName="revoke-keys"
Jul 17 10:23:56.377943 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.377872    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="db0a1d18-6c8b-45de-bb7d-fd10a56367ab" containerName="revoke-keys"
Jul 17 10:23:56.377943 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.377880    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="db0a1d18-6c8b-45de-bb7d-fd10a56367ab" containerName="revoke-keys"
Jul 17 10:23:56.377943 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.377886    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="db0a1d18-6c8b-45de-bb7d-fd10a56367ab" containerName="revoke-keys"
Jul 17 10:23:56.377943 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.377892    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf340eb4-e72a-4fe8-b125-158eb16e0543" containerName="istio-proxy"
Jul 17 10:23:56.377943 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.377898    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf340eb4-e72a-4fe8-b125-158eb16e0543" containerName="istio-proxy"
Jul 17 10:23:56.377943 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.377904    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="af4da765-16c1-4c33-aa27-d53b2d6bf53a" containerName="revoke-keys"
Jul 17 10:23:56.381323 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.381304    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vkpq8"
Jul 17 10:23:56.384350 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.384317    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:23:56.387853 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.387826    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vkpq8"]
Jul 17 10:23:56.418985 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.418947    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb300937-19ee-4391-8211-f8dfc503d640-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-vkpq8\" (UID: \"bb300937-19ee-4391-8211-f8dfc503d640\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vkpq8"
Jul 17 10:23:56.422973 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.422944    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-x5rwn_aff066d2-5135-49e2-b041-39424acf8fb6/istio-proxy/1.log"
Jul 17 10:23:56.520116 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.520077    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb300937-19ee-4391-8211-f8dfc503d640-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-vkpq8\" (UID: \"bb300937-19ee-4391-8211-f8dfc503d640\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vkpq8"
Jul 17 10:23:56.520784 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.520754    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb300937-19ee-4391-8211-f8dfc503d640-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-125842b9-vkpq8\" (UID: \"bb300937-19ee-4391-8211-f8dfc503d640\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vkpq8"
Jul 17 10:23:56.562009 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.561913    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-wpz7b_73bdc8e2-2f1e-41df-8396-cea33c4fd3fd/payload-pre-processing/0.log"
Jul 17 10:23:56.687615 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.687580    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-9b5xh_40ad3c79-63ba-46bd-ab13-97648ab5d574/payload-processing/0.log"
Jul 17 10:23:56.693675 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.693650    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vkpq8"
Jul 17 10:23:56.845690 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.845443    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vkpq8"]
Jul 17 10:23:56.848778 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:23:56.848751    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb300937_19ee_4391_8211_f8dfc503d640.slice/crio-7ed12827f87c77713a8f0ac4bde444725b90a8f7d9cb2e678c8d91d689895462 WatchSource:0}: Error finding container 7ed12827f87c77713a8f0ac4bde444725b90a8f7d9cb2e678c8d91d689895462: Status 404 returned error can't find the container with id 7ed12827f87c77713a8f0ac4bde444725b90a8f7d9cb2e678c8d91d689895462
Jul 17 10:23:56.929642 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:56.929609    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vkpq8" event={"ID":"bb300937-19ee-4391-8211-f8dfc503d640","Type":"ContainerStarted","Data":"7ed12827f87c77713a8f0ac4bde444725b90a8f7d9cb2e678c8d91d689895462"}
Jul 17 10:23:57.186957 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:57.186921    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-svjd5_4c8c40db-54fe-41b4-a87c-5ce9f5c4d5c2/main/0.log"
Jul 17 10:23:57.442136 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:57.442111    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-nt9tk_fa111f9b-d374-4991-a041-8c05c7c45011/main/0.log"
Jul 17 10:23:57.939146 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:57.939111    2574 generic.go:356] "Generic (PLEG): container finished" podID="bb300937-19ee-4391-8211-f8dfc503d640" containerID="3448aecff42bcd4caf7e05dc6a14b0a5afb21ad0c38c6e7cdaf08e6da16e1441" exitCode=6
Jul 17 10:23:57.939362 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:57.939254    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vkpq8" event={"ID":"bb300937-19ee-4391-8211-f8dfc503d640","Type":"ContainerDied","Data":"3448aecff42bcd4caf7e05dc6a14b0a5afb21ad0c38c6e7cdaf08e6da16e1441"}
Jul 17 10:23:57.939441 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:57.939410    2574 scope.go:117] "RemoveContainer" containerID="3448aecff42bcd4caf7e05dc6a14b0a5afb21ad0c38c6e7cdaf08e6da16e1441"
Jul 17 10:23:58.947233 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:58.947195    2574 generic.go:356] "Generic (PLEG): container finished" podID="bb300937-19ee-4391-8211-f8dfc503d640" containerID="da7e05f0f0cf32b5b10d655f3960c1f360121e45b23d1664b34fc3588eb7ab27" exitCode=6
Jul 17 10:23:58.947762 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:58.947278    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vkpq8" event={"ID":"bb300937-19ee-4391-8211-f8dfc503d640","Type":"ContainerDied","Data":"da7e05f0f0cf32b5b10d655f3960c1f360121e45b23d1664b34fc3588eb7ab27"}
Jul 17 10:23:58.947762 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:58.947356    2574 scope.go:117] "RemoveContainer" containerID="3448aecff42bcd4caf7e05dc6a14b0a5afb21ad0c38c6e7cdaf08e6da16e1441"
Jul 17 10:23:58.947762 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:58.947513    2574 scope.go:117] "RemoveContainer" containerID="da7e05f0f0cf32b5b10d655f3960c1f360121e45b23d1664b34fc3588eb7ab27"
Jul 17 10:23:58.947762 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:23:58.947744    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-vkpq8_odh-ai-gateway-infra(bb300937-19ee-4391-8211-f8dfc503d640)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vkpq8" podUID="bb300937-19ee-4391-8211-f8dfc503d640"
Jul 17 10:23:59.955109 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:23:59.955077    2574 scope.go:117] "RemoveContainer" containerID="da7e05f0f0cf32b5b10d655f3960c1f360121e45b23d1664b34fc3588eb7ab27"
Jul 17 10:23:59.955558 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:23:59.955327    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-vkpq8_odh-ai-gateway-infra(bb300937-19ee-4391-8211-f8dfc503d640)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vkpq8" podUID="bb300937-19ee-4391-8211-f8dfc503d640"
Jul 17 10:24:07.562553 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:07.562522    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-trvgv_f357001d-cac3-4c7d-b427-7cc231e3d4ff/global-pull-secret-syncer/0.log"
Jul 17 10:24:07.662356 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:07.662320    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-gd59l_0f1dd372-9aca-4b16-8020-eee6e421e6a2/konnectivity-agent/0.log"
Jul 17 10:24:07.755281 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:07.755225    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-235.ec2.internal_fa208b626e54278e56162a86ee4791d7/haproxy/0.log"
Jul 17 10:24:13.371828 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:13.371795    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-mk4bq_8e21489e-210e-4b51-9777-e637fb5cfc74/manager/0.log"
Jul 17 10:24:13.400860 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:13.400832    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-zvqp9_0e5c29c3-e1f5-48b8-a3a1-f3579196c61c/manager/0.log"
Jul 17 10:24:13.476773 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:13.476715    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-5x7hd_87acf608-ce98-43fc-820d-11da3b3cbc0d/registry-server/0.log"
Jul 17 10:24:13.562556 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:13.562520    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-k8fhl_bcd494e5-0002-47a1-9c6d-c31af2e510e2/manager/0.log"
Jul 17 10:24:13.652106 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:13.652079    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-hrbff_7eceeaf3-ab95-4000-913c-2254baf18b00/manager/0.log"
Jul 17 10:24:14.901725 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:14.901695    2574 scope.go:117] "RemoveContainer" containerID="da7e05f0f0cf32b5b10d655f3960c1f360121e45b23d1664b34fc3588eb7ab27"
Jul 17 10:24:15.614947 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:15.614928    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-64874587b8-krgcd_d8bfa5bb-c0f1-41be-8faf-c6bd5206f92b/metrics-server/0.log"
Jul 17 10:24:15.686664 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:15.686634    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bmcbf_c644d32e-0c8d-403f-ae35-5d7cff59d490/node-exporter/0.log"
Jul 17 10:24:15.712615 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:15.712592    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bmcbf_c644d32e-0c8d-403f-ae35-5d7cff59d490/kube-rbac-proxy/0.log"
Jul 17 10:24:15.745353 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:15.745317    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bmcbf_c644d32e-0c8d-403f-ae35-5d7cff59d490/init-textfile/0.log"
Jul 17 10:24:15.935693 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:15.935662    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-b5n86_200c4e96-7d88-49ff-8990-8bd65d3c2817/kube-rbac-proxy-main/0.log"
Jul 17 10:24:15.961929 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:15.961904    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-b5n86_200c4e96-7d88-49ff-8990-8bd65d3c2817/kube-rbac-proxy-self/0.log"
Jul 17 10:24:15.987264 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:15.987207    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-b5n86_200c4e96-7d88-49ff-8990-8bd65d3c2817/openshift-state-metrics/0.log"
Jul 17 10:24:16.028383 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:16.028360    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_409e5645-8be5-4081-9459-e01da7957cfb/prometheus/0.log"
Jul 17 10:24:16.049701 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:16.049669    2574 generic.go:356] "Generic (PLEG): container finished" podID="bb300937-19ee-4391-8211-f8dfc503d640" containerID="74638fafa3d5c3dafc0eaa6374d08ed1bc584066f176334b7cab8b582ee32df1" exitCode=6
Jul 17 10:24:16.049852 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:16.049735    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vkpq8" event={"ID":"bb300937-19ee-4391-8211-f8dfc503d640","Type":"ContainerDied","Data":"74638fafa3d5c3dafc0eaa6374d08ed1bc584066f176334b7cab8b582ee32df1"}
Jul 17 10:24:16.049852 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:16.049805    2574 scope.go:117] "RemoveContainer" containerID="da7e05f0f0cf32b5b10d655f3960c1f360121e45b23d1664b34fc3588eb7ab27"
Jul 17 10:24:16.050154 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:16.050137    2574 scope.go:117] "RemoveContainer" containerID="74638fafa3d5c3dafc0eaa6374d08ed1bc584066f176334b7cab8b582ee32df1"
Jul 17 10:24:16.050437 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:24:16.050411    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 20s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-125842b9-vkpq8_odh-ai-gateway-infra(bb300937-19ee-4391-8211-f8dfc503d640)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vkpq8" podUID="bb300937-19ee-4391-8211-f8dfc503d640"
Jul 17 10:24:16.051223 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:16.051086    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_409e5645-8be5-4081-9459-e01da7957cfb/config-reloader/0.log"
Jul 17 10:24:16.080851 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:16.080829    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_409e5645-8be5-4081-9459-e01da7957cfb/thanos-sidecar/0.log"
Jul 17 10:24:16.105146 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:16.105124    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_409e5645-8be5-4081-9459-e01da7957cfb/kube-rbac-proxy-web/0.log"
Jul 17 10:24:16.128967 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:16.128934    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_409e5645-8be5-4081-9459-e01da7957cfb/kube-rbac-proxy/0.log"
Jul 17 10:24:16.151230 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:16.151207    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_409e5645-8be5-4081-9459-e01da7957cfb/kube-rbac-proxy-thanos/0.log"
Jul 17 10:24:16.177734 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:16.177698    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_409e5645-8be5-4081-9459-e01da7957cfb/init-config-reloader/0.log"
Jul 17 10:24:16.207724 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:16.207666    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-ctl9f_7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e/prometheus-operator/0.log"
Jul 17 10:24:16.230715 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:16.230693    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-ctl9f_7e4f20db-51a2-4b33-a7c1-bdbdbfd4eb0e/kube-rbac-proxy/0.log"
Jul 17 10:24:16.260907 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:16.260873    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-sm7n2_0cc618f8-0247-4eec-ae71-4a69b6495872/prometheus-operator-admission-webhook/0.log"
Jul 17 10:24:16.298044 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:16.298022    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6966f9675-gcrpl_c6c7a08a-8f34-41b0-99cd-9613bf032f67/telemeter-client/0.log"
Jul 17 10:24:16.321780 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:16.321747    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6966f9675-gcrpl_c6c7a08a-8f34-41b0-99cd-9613bf032f67/reload/0.log"
Jul 17 10:24:16.358616 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:16.358566    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6966f9675-gcrpl_c6c7a08a-8f34-41b0-99cd-9613bf032f67/kube-rbac-proxy/0.log"
Jul 17 10:24:17.088732 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:17.088695    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vkpq8"]
Jul 17 10:24:17.245373 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:17.245349    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vkpq8"
Jul 17 10:24:17.325468 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:17.325434    2574 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb300937-19ee-4391-8211-f8dfc503d640-maas-api-service-ca\") pod \"bb300937-19ee-4391-8211-f8dfc503d640\" (UID: \"bb300937-19ee-4391-8211-f8dfc503d640\") "
Jul 17 10:24:17.325805 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:17.325783    2574 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb300937-19ee-4391-8211-f8dfc503d640-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bb300937-19ee-4391-8211-f8dfc503d640" (UID: "bb300937-19ee-4391-8211-f8dfc503d640"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 17 10:24:17.427110 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:17.427067    2574 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb300937-19ee-4391-8211-f8dfc503d640-maas-api-service-ca\") on node \"ip-10-0-136-235.ec2.internal\" DevicePath \"\""
Jul 17 10:24:18.029284 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.029250    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5lvm5/perf-node-gather-daemonset-29ttm"]
Jul 17 10:24:18.029928 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.029905    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb300937-19ee-4391-8211-f8dfc503d640" containerName="revoke-keys"
Jul 17 10:24:18.029928 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.029929    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb300937-19ee-4391-8211-f8dfc503d640" containerName="revoke-keys"
Jul 17 10:24:18.030061 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.029940    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db0a1d18-6c8b-45de-bb7d-fd10a56367ab" containerName="revoke-keys"
Jul 17 10:24:18.030061 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.029945    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="db0a1d18-6c8b-45de-bb7d-fd10a56367ab" containerName="revoke-keys"
Jul 17 10:24:18.030061 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.029966    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb300937-19ee-4391-8211-f8dfc503d640" containerName="revoke-keys"
Jul 17 10:24:18.030061 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.029972    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb300937-19ee-4391-8211-f8dfc503d640" containerName="revoke-keys"
Jul 17 10:24:18.030061 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.030033    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb300937-19ee-4391-8211-f8dfc503d640" containerName="revoke-keys"
Jul 17 10:24:18.030061 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.030043    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb300937-19ee-4391-8211-f8dfc503d640" containerName="revoke-keys"
Jul 17 10:24:18.030061 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.030051    2574 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb300937-19ee-4391-8211-f8dfc503d640" containerName="revoke-keys"
Jul 17 10:24:18.033212 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.033196    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-29ttm"
Jul 17 10:24:18.036479 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.036458    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5lvm5\"/\"openshift-service-ca.crt\""
Jul 17 10:24:18.036570 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.036501    2574 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5lvm5\"/\"kube-root-ca.crt\""
Jul 17 10:24:18.037859 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.037841    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5lvm5\"/\"default-dockercfg-vqkh2\""
Jul 17 10:24:18.043776 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.043754    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5lvm5/perf-node-gather-daemonset-29ttm"]
Jul 17 10:24:18.071023 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.070994    2574 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vkpq8"
Jul 17 10:24:18.071023 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.071022    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vkpq8" event={"ID":"bb300937-19ee-4391-8211-f8dfc503d640","Type":"ContainerDied","Data":"7ed12827f87c77713a8f0ac4bde444725b90a8f7d9cb2e678c8d91d689895462"}
Jul 17 10:24:18.071281 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.071075    2574 scope.go:117] "RemoveContainer" containerID="74638fafa3d5c3dafc0eaa6374d08ed1bc584066f176334b7cab8b582ee32df1"
Jul 17 10:24:18.092888 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.092858    2574 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vkpq8"]
Jul 17 10:24:18.104713 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.104692    2574 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-125842b9-vkpq8"]
Jul 17 10:24:18.134401 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.134361    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1-podres\") pod \"perf-node-gather-daemonset-29ttm\" (UID: \"5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-29ttm"
Jul 17 10:24:18.134579 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.134429    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqvms\" (UniqueName: \"kubernetes.io/projected/5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1-kube-api-access-pqvms\") pod \"perf-node-gather-daemonset-29ttm\" (UID: \"5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-29ttm"
Jul 17 10:24:18.134579 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.134454    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1-lib-modules\") pod \"perf-node-gather-daemonset-29ttm\" (UID: \"5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-29ttm"
Jul 17 10:24:18.134579 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.134515    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1-sys\") pod \"perf-node-gather-daemonset-29ttm\" (UID: \"5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-29ttm"
Jul 17 10:24:18.134579 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.134557    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1-proc\") pod \"perf-node-gather-daemonset-29ttm\" (UID: \"5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-29ttm"
Jul 17 10:24:18.235436 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.235406    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1-sys\") pod \"perf-node-gather-daemonset-29ttm\" (UID: \"5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-29ttm"
Jul 17 10:24:18.235613 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.235441    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1-proc\") pod \"perf-node-gather-daemonset-29ttm\" (UID: \"5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-29ttm"
Jul 17 10:24:18.235613 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.235507    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1-podres\") pod \"perf-node-gather-daemonset-29ttm\" (UID: \"5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-29ttm"
Jul 17 10:24:18.235613 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.235533    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1-sys\") pod \"perf-node-gather-daemonset-29ttm\" (UID: \"5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-29ttm"
Jul 17 10:24:18.235613 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.235552    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqvms\" (UniqueName: \"kubernetes.io/projected/5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1-kube-api-access-pqvms\") pod \"perf-node-gather-daemonset-29ttm\" (UID: \"5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-29ttm"
Jul 17 10:24:18.235613 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.235577    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1-lib-modules\") pod \"perf-node-gather-daemonset-29ttm\" (UID: \"5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-29ttm"
Jul 17 10:24:18.235613 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.235594    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1-proc\") pod \"perf-node-gather-daemonset-29ttm\" (UID: \"5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-29ttm"
Jul 17 10:24:18.235811 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.235658    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1-podres\") pod \"perf-node-gather-daemonset-29ttm\" (UID: \"5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-29ttm"
Jul 17 10:24:18.235811 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.235701    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1-lib-modules\") pod \"perf-node-gather-daemonset-29ttm\" (UID: \"5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-29ttm"
Jul 17 10:24:18.244862 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.244837    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqvms\" (UniqueName: \"kubernetes.io/projected/5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1-kube-api-access-pqvms\") pod \"perf-node-gather-daemonset-29ttm\" (UID: \"5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1\") " pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-29ttm"
Jul 17 10:24:18.343741 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.343660    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-29ttm"
Jul 17 10:24:18.487061 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.487032    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5lvm5/perf-node-gather-daemonset-29ttm"]
Jul 17 10:24:18.488551 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:24:18.488526    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5b40b0a4_c57a_4d49_8d42_4d9b5e4ac4b1.slice/crio-81c8248618101ad4a7426bf335fdc87062dfada772c7b11bb4856a4e5353cf20 WatchSource:0}: Error finding container 81c8248618101ad4a7426bf335fdc87062dfada772c7b11bb4856a4e5353cf20: Status 404 returned error can't find the container with id 81c8248618101ad4a7426bf335fdc87062dfada772c7b11bb4856a4e5353cf20
Jul 17 10:24:18.816066 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.816017    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7464459688-4rdzs_5c2d4229-4cd7-4951-b0d3-43441e51732d/console/0.log"
Jul 17 10:24:18.853443 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.853412    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-hjfk5_21d479b5-ee17-4fb5-acf9-541487651af1/download-server/0.log"
Jul 17 10:24:18.898407 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:18.898327    2574 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb300937-19ee-4391-8211-f8dfc503d640" path="/var/lib/kubelet/pods/bb300937-19ee-4391-8211-f8dfc503d640/volumes"
Jul 17 10:24:19.079514 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:19.079478    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-29ttm" event={"ID":"5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1","Type":"ContainerStarted","Data":"8d10154ac79ddd9755fd9cac9e6f599217729dd4a10916f9b7e96d8dbc430a26"}
Jul 17 10:24:19.079708 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:19.079528    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-29ttm" event={"ID":"5b40b0a4-c57a-4d49-8d42-4d9b5e4ac4b1","Type":"ContainerStarted","Data":"81c8248618101ad4a7426bf335fdc87062dfada772c7b11bb4856a4e5353cf20"}
Jul 17 10:24:19.079774 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:19.079720    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-29ttm"
Jul 17 10:24:19.098185 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:19.098141    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-29ttm" podStartSLOduration=1.098126717 podStartE2EDuration="1.098126717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 10:24:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:24:19.095426613 +0000 UTC m=+3456.678064991" watchObservedRunningTime="2026-07-17 10:24:19.098126717 +0000 UTC m=+3456.680765094"
Jul 17 10:24:20.561360 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:20.561324    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gdlqf_fdac6d65-9077-4012-bcfb-b19556f35d5f/dns/0.log"
Jul 17 10:24:20.582802 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:20.582776    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gdlqf_fdac6d65-9077-4012-bcfb-b19556f35d5f/kube-rbac-proxy/0.log"
Jul 17 10:24:20.606897 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:20.606868    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fz9bt_eccc078f-2e4c-4a45-9056-d0cd7053faf6/dns-node-resolver/0.log"
Jul 17 10:24:21.228146 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:21.228115    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4x7kz_abec7743-4f1a-45ad-bddf-3f9c0b7d42dc/node-ca/0.log"
Jul 17 10:24:22.519269 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:22.519209    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-546bc49978-vw2hd_1f7ee6c1-0dfe-4612-93a9-beccaa966946/kube-auth-proxy/0.log"
Jul 17 10:24:22.552973 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:22.552941    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-x5rwn_aff066d2-5135-49e2-b041-39424acf8fb6/istio-proxy/0.log"
Jul 17 10:24:22.584651 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:22.584621    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-x5rwn_aff066d2-5135-49e2-b041-39424acf8fb6/istio-proxy/1.log"
Jul 17 10:24:22.622261 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:22.622216    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-wpz7b_73bdc8e2-2f1e-41df-8396-cea33c4fd3fd/payload-pre-processing/0.log"
Jul 17 10:24:22.652077 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:22.652052    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-9b5xh_40ad3c79-63ba-46bd-ab13-97648ab5d574/payload-processing/0.log"
Jul 17 10:24:23.429482 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:23.429450    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vqvdn_78c11134-f4c9-4e8a-b87c-3ea12f21e6d1/serve-healthcheck-canary/0.log"
Jul 17 10:24:24.100346 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:24.100279    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zvktb_44a47c07-93d4-4047-9e80-3ffa7f6b066e/kube-rbac-proxy/0.log"
Jul 17 10:24:24.125825 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:24.125796    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zvktb_44a47c07-93d4-4047-9e80-3ffa7f6b066e/exporter/0.log"
Jul 17 10:24:24.151919 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:24.151890    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zvktb_44a47c07-93d4-4047-9e80-3ffa7f6b066e/extractor/0.log"
Jul 17 10:24:25.096421 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:25.096390    2574 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5lvm5/perf-node-gather-daemonset-29ttm"
Jul 17 10:24:26.235985 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:26.235950    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-24c6h_75da319e-cb7b-471b-811c-cf9f9fbf120a/manager/0.log"
Jul 17 10:24:26.372126 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:26.372095    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-v9rhs_03e554d1-089a-4801-aa0c-ace3e09085a2/manager/0.log"
Jul 17 10:24:26.483866 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:26.483826    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6c5c6b98f-rlrg6_8ab844e0-7c94-4003-9e42-ada02b05d4e9/manager/0.log"
Jul 17 10:24:26.506413 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:26.506314    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-96757_e56c1862-83dd-417d-bd30-60abc092e276/server/0.log"
Jul 17 10:24:26.577918 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:26.577876    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-bs5g8_2cdf35a8-c7a2-49c1-98d5-7153f317c517/manager/0.log"
Jul 17 10:24:26.766387 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:26.766261    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b5566fdf9-ndp9h_ce30ed03-36b2-442a-8dba-13d434d42d39/manager/0.log"
Jul 17 10:24:26.788575 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:26.788546    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b5566fdf9-ndp9h_ce30ed03-36b2-442a-8dba-13d434d42d39/copy-manifests/0.log"
Jul 17 10:24:26.921574 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:26.921539    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s599v"]
Jul 17 10:24:26.922016 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:26.922001    2574 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb300937-19ee-4391-8211-f8dfc503d640" containerName="revoke-keys"
Jul 17 10:24:26.922069 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:26.922018    2574 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb300937-19ee-4391-8211-f8dfc503d640" containerName="revoke-keys"
Jul 17 10:24:26.925318 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:26.925300    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s599v"
Jul 17 10:24:26.929526 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:26.928756    2574 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pkn2b\""
Jul 17 10:24:26.931736 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:26.931708    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s599v"]
Jul 17 10:24:27.021276 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:27.021150    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbf8cb37-99df-42ee-a1d9-afe87cca6c72-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-s599v\" (UID: \"dbf8cb37-99df-42ee-a1d9-afe87cca6c72\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s599v"
Jul 17 10:24:27.122768 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:27.122732    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbf8cb37-99df-42ee-a1d9-afe87cca6c72-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-s599v\" (UID: \"dbf8cb37-99df-42ee-a1d9-afe87cca6c72\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s599v"
Jul 17 10:24:27.123624 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:27.123598    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbf8cb37-99df-42ee-a1d9-afe87cca6c72-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-10ce7fe6-s599v\" (UID: \"dbf8cb37-99df-42ee-a1d9-afe87cca6c72\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s599v"
Jul 17 10:24:27.239844 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:27.239808    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s599v"
Jul 17 10:24:27.392416 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:27.392386    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s599v"]
Jul 17 10:24:27.394210 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:24:27.394179    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbf8cb37_99df_42ee_a1d9_afe87cca6c72.slice/crio-81fc5cdc9f5b4446275d6a16c79671b45c36d4126d74474c10dd95ca744713f8 WatchSource:0}: Error finding container 81fc5cdc9f5b4446275d6a16c79671b45c36d4126d74474c10dd95ca744713f8: Status 404 returned error can't find the container with id 81fc5cdc9f5b4446275d6a16c79671b45c36d4126d74474c10dd95ca744713f8
Jul 17 10:24:28.134454 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:28.134424    2574 generic.go:356] "Generic (PLEG): container finished" podID="dbf8cb37-99df-42ee-a1d9-afe87cca6c72" containerID="916608ed13d4c6262c2a70deab3eb72d3b39f860566396edbd9cb299dbf205e5" exitCode=6
Jul 17 10:24:28.134656 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:28.134501    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s599v" event={"ID":"dbf8cb37-99df-42ee-a1d9-afe87cca6c72","Type":"ContainerDied","Data":"916608ed13d4c6262c2a70deab3eb72d3b39f860566396edbd9cb299dbf205e5"}
Jul 17 10:24:28.134656 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:28.134553    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s599v" event={"ID":"dbf8cb37-99df-42ee-a1d9-afe87cca6c72","Type":"ContainerStarted","Data":"81fc5cdc9f5b4446275d6a16c79671b45c36d4126d74474c10dd95ca744713f8"}
Jul 17 10:24:28.134767 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:28.134729    2574 scope.go:117] "RemoveContainer" containerID="916608ed13d4c6262c2a70deab3eb72d3b39f860566396edbd9cb299dbf205e5"
Jul 17 10:24:28.288421 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:28.288383    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-nzndn_765b8fd3-de37-4e60-84ea-54f2007bbf22/openshift-lws-operator/0.log"
Jul 17 10:24:28.662972 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:28.662879    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-546bc49978-vw2hd_1f7ee6c1-0dfe-4612-93a9-beccaa966946/kube-auth-proxy/0.log"
Jul 17 10:24:28.683692 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:28.683661    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-x5rwn_aff066d2-5135-49e2-b041-39424acf8fb6/istio-proxy/0.log"
Jul 17 10:24:28.720767 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:28.720726    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-x5rwn_aff066d2-5135-49e2-b041-39424acf8fb6/istio-proxy/1.log"
Jul 17 10:24:28.739311 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:28.739277    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-wpz7b_73bdc8e2-2f1e-41df-8396-cea33c4fd3fd/payload-pre-processing/0.log"
Jul 17 10:24:28.758746 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:28.758716    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-9b5xh_40ad3c79-63ba-46bd-ab13-97648ab5d574/payload-processing/0.log"
Jul 17 10:24:29.052078 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:29.051986    2574 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-5nm8l"]
Jul 17 10:24:29.056023 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:29.056003    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-5nm8l"
Jul 17 10:24:29.063362 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:29.063329    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-5nm8l"]
Jul 17 10:24:29.142105 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:29.142068    2574 generic.go:356] "Generic (PLEG): container finished" podID="dbf8cb37-99df-42ee-a1d9-afe87cca6c72" containerID="a73f9d1a2822219c048fdddf2be0934c16f8d8e8b795a28a2c888cc324e8a129" exitCode=6
Jul 17 10:24:29.143009 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:29.142163    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s599v" event={"ID":"dbf8cb37-99df-42ee-a1d9-afe87cca6c72","Type":"ContainerDied","Data":"a73f9d1a2822219c048fdddf2be0934c16f8d8e8b795a28a2c888cc324e8a129"}
Jul 17 10:24:29.143009 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:29.142228    2574 scope.go:117] "RemoveContainer" containerID="916608ed13d4c6262c2a70deab3eb72d3b39f860566396edbd9cb299dbf205e5"
Jul 17 10:24:29.143009 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:29.142562    2574 scope.go:117] "RemoveContainer" containerID="a73f9d1a2822219c048fdddf2be0934c16f8d8e8b795a28a2c888cc324e8a129"
Jul 17 10:24:29.143009 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:24:29.142814    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-s599v_odh-ai-gateway-infra(dbf8cb37-99df-42ee-a1d9-afe87cca6c72)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s599v" podUID="dbf8cb37-99df-42ee-a1d9-afe87cca6c72"
Jul 17 10:24:29.144054 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:29.143811    2574 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f749064-47e0-4419-8589-4ac3d0306e99-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-5nm8l\" (UID: \"1f749064-47e0-4419-8589-4ac3d0306e99\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-5nm8l"
Jul 17 10:24:29.244986 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:29.244928    2574 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f749064-47e0-4419-8589-4ac3d0306e99-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-5nm8l\" (UID: \"1f749064-47e0-4419-8589-4ac3d0306e99\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-5nm8l"
Jul 17 10:24:29.245633 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:29.245613    2574 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f749064-47e0-4419-8589-4ac3d0306e99-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3fc19893-5nm8l\" (UID: \"1f749064-47e0-4419-8589-4ac3d0306e99\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-5nm8l"
Jul 17 10:24:29.370042 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:29.369950    2574 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-5nm8l"
Jul 17 10:24:29.727552 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:29.727525    2574 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-5nm8l"]
Jul 17 10:24:29.729437 ip-10-0-136-235 kubenswrapper[2574]: W0717 10:24:29.729409    2574 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f749064_47e0_4419_8589_4ac3d0306e99.slice/crio-a0d31798d08fedd449c8d8bde6d9fafeb952e0f9f1b874a366dcf1cb3f882ee5 WatchSource:0}: Error finding container a0d31798d08fedd449c8d8bde6d9fafeb952e0f9f1b874a366dcf1cb3f882ee5: Status 404 returned error can't find the container with id a0d31798d08fedd449c8d8bde6d9fafeb952e0f9f1b874a366dcf1cb3f882ee5
Jul 17 10:24:30.150474 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:30.150384    2574 scope.go:117] "RemoveContainer" containerID="a73f9d1a2822219c048fdddf2be0934c16f8d8e8b795a28a2c888cc324e8a129"
Jul 17 10:24:30.150696 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:24:30.150674    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-10ce7fe6-s599v_odh-ai-gateway-infra(dbf8cb37-99df-42ee-a1d9-afe87cca6c72)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-10ce7fe6-s599v" podUID="dbf8cb37-99df-42ee-a1d9-afe87cca6c72"
Jul 17 10:24:30.151702 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:30.151678    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-5nm8l" event={"ID":"1f749064-47e0-4419-8589-4ac3d0306e99","Type":"ContainerStarted","Data":"bca7a55755478202d1b688f5059c532a676a59a2a94e8cb1e5072b3c6977db4d"}
Jul 17 10:24:30.151798 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:30.151736    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-5nm8l" event={"ID":"1f749064-47e0-4419-8589-4ac3d0306e99","Type":"ContainerStarted","Data":"a0d31798d08fedd449c8d8bde6d9fafeb952e0f9f1b874a366dcf1cb3f882ee5"}
Jul 17 10:24:30.187904 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:30.187848    2574 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-5nm8l" podStartSLOduration=1.187830141 podStartE2EDuration="1.187830141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 10:24:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 10:24:30.185298751 +0000 UTC m=+3467.767937128" watchObservedRunningTime="2026-07-17 10:24:30.187830141 +0000 UTC m=+3467.770468538"
Jul 17 10:24:30.253849 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:30.253816    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-546bc49978-vw2hd_1f7ee6c1-0dfe-4612-93a9-beccaa966946/kube-auth-proxy/0.log"
Jul 17 10:24:30.276117 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:30.276080    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-x5rwn_aff066d2-5135-49e2-b041-39424acf8fb6/istio-proxy/0.log"
Jul 17 10:24:30.309547 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:30.309511    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-x5rwn_aff066d2-5135-49e2-b041-39424acf8fb6/istio-proxy/1.log"
Jul 17 10:24:30.328821 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:30.328786    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-wpz7b_73bdc8e2-2f1e-41df-8396-cea33c4fd3fd/payload-pre-processing/0.log"
Jul 17 10:24:30.350905 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:30.350873    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-9b5xh_40ad3c79-63ba-46bd-ab13-97648ab5d574/payload-processing/0.log"
Jul 17 10:24:31.159192 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:31.159158    2574 generic.go:356] "Generic (PLEG): container finished" podID="1f749064-47e0-4419-8589-4ac3d0306e99" containerID="bca7a55755478202d1b688f5059c532a676a59a2a94e8cb1e5072b3c6977db4d" exitCode=6
Jul 17 10:24:31.159629 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:31.159235    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-5nm8l" event={"ID":"1f749064-47e0-4419-8589-4ac3d0306e99","Type":"ContainerDied","Data":"bca7a55755478202d1b688f5059c532a676a59a2a94e8cb1e5072b3c6977db4d"}
Jul 17 10:24:31.159629 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:31.159467    2574 scope.go:117] "RemoveContainer" containerID="bca7a55755478202d1b688f5059c532a676a59a2a94e8cb1e5072b3c6977db4d"
Jul 17 10:24:32.166917 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:32.166876    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-5nm8l" event={"ID":"1f749064-47e0-4419-8589-4ac3d0306e99","Type":"ContainerStarted","Data":"482923ed77c08e205977d1bd3aa29ab2a113c2579bf1b56cbba5827392c89798"}
Jul 17 10:24:33.174287 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:33.174226    2574 generic.go:356] "Generic (PLEG): container finished" podID="1f749064-47e0-4419-8589-4ac3d0306e99" containerID="482923ed77c08e205977d1bd3aa29ab2a113c2579bf1b56cbba5827392c89798" exitCode=6
Jul 17 10:24:33.174287 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:33.174272    2574 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-5nm8l" event={"ID":"1f749064-47e0-4419-8589-4ac3d0306e99","Type":"ContainerDied","Data":"482923ed77c08e205977d1bd3aa29ab2a113c2579bf1b56cbba5827392c89798"}
Jul 17 10:24:33.174817 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:33.174350    2574 scope.go:117] "RemoveContainer" containerID="bca7a55755478202d1b688f5059c532a676a59a2a94e8cb1e5072b3c6977db4d"
Jul 17 10:24:33.174817 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:33.174577    2574 scope.go:117] "RemoveContainer" containerID="482923ed77c08e205977d1bd3aa29ab2a113c2579bf1b56cbba5827392c89798"
Jul 17 10:24:33.174817 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:24:33.174795    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-5nm8l_odh-ai-gateway-infra(1f749064-47e0-4419-8589-4ac3d0306e99)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-5nm8l" podUID="1f749064-47e0-4419-8589-4ac3d0306e99"
Jul 17 10:24:33.669996 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:33.669935    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-jxtbz_1931d80b-6799-4456-a479-23d203f2070a/kube-storage-version-migrator-operator/1.log"
Jul 17 10:24:33.670698 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:33.670672    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-jxtbz_1931d80b-6799-4456-a479-23d203f2070a/kube-storage-version-migrator-operator/0.log"
Jul 17 10:24:34.182533 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:34.182499    2574 scope.go:117] "RemoveContainer" containerID="482923ed77c08e205977d1bd3aa29ab2a113c2579bf1b56cbba5827392c89798"
Jul 17 10:24:34.183021 ip-10-0-136-235 kubenswrapper[2574]: E0717 10:24:34.182798    2574 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3fc19893-5nm8l_odh-ai-gateway-infra(1f749064-47e0-4419-8589-4ac3d0306e99)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3fc19893-5nm8l" podUID="1f749064-47e0-4419-8589-4ac3d0306e99"
Jul 17 10:24:34.940491 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:34.940459    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-d4ksz_5c165f52-605b-48e9-9cff-bf1e0b7b97af/kube-multus-additional-cni-plugins/0.log"
Jul 17 10:24:34.964182 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:34.964157    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-d4ksz_5c165f52-605b-48e9-9cff-bf1e0b7b97af/egress-router-binary-copy/0.log"
Jul 17 10:24:34.990935 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:34.990905    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-d4ksz_5c165f52-605b-48e9-9cff-bf1e0b7b97af/cni-plugins/0.log"
Jul 17 10:24:35.014976 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:35.014938    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-d4ksz_5c165f52-605b-48e9-9cff-bf1e0b7b97af/bond-cni-plugin/0.log"
Jul 17 10:24:35.041323 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:35.041297    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-d4ksz_5c165f52-605b-48e9-9cff-bf1e0b7b97af/routeoverride-cni/0.log"
Jul 17 10:24:35.064981 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:35.064953    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-d4ksz_5c165f52-605b-48e9-9cff-bf1e0b7b97af/whereabouts-cni-bincopy/0.log"
Jul 17 10:24:35.088227 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:35.088199    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-d4ksz_5c165f52-605b-48e9-9cff-bf1e0b7b97af/whereabouts-cni/0.log"
Jul 17 10:24:35.322515 ip-10-0-136-235 kubenswrapper[2574]: I0717 10:24:35.322428    2574 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mcbxh_87fc473b-6f15-4afc-8632-62937f882dbc/kube-multus/0.log"